Title: Batch Bayesian Optimization For Replicable Experimental Design
Venue: NeurIPS 2023
Accepted: yes
Keywords: Bayesian optimization

Gaussian processes

AI4Science
Abstract: Many real-world experimental design problems (a) evaluate multiple experimental conditions in parallel and (b) replicate each condition multiple times due to large and heteroscedastic observation noise. Given a fixed total budget, this naturally induces a trade-off between evaluating more unique conditions while replicating each of them fewer times vs. evaluating fewer unique conditions and replicating each more times. Moreover, in these problems, practitioners may be risk-averse and hence prefer an input with both good average performance and small variability. To tackle both challenges, we propose the Batch Thompson Sampling for Replicable Experimental Design (BTS-RED) framework, which encompasses three algorithms. Our BTS-RED-Known and BTS-RED-Unknown algorithms, for, respectively, known and unknown noise variance, choose the number of replications adaptively rather than deterministically such that an input with a larger noise variance is replicated more times. As a result, despite the noise heteroscedasticity, both algorithms enjoy a theoretical guarantee and are asymptotically no-regret. Our Mean-Var-BTS-RED algorithm aims at risk-averse optimization and is also asymptotically no-regret. We also show the effectiveness of our algorithms in two practical real-world applications: precision agriculture and AutoML.

Batch Bayesian Optimization for Replicable Experimental Design

Abstract

Many real-world experimental design problems (a) evaluate multiple experimental conditions in parallel and (b) replicate each condition multiple times due to large and heteroscedastic observation noise. Given a fixed total budget, this naturally induces a trade-off between evaluating more unique conditions while replicating each of them fewer times vs. evaluating fewer unique conditions and replicating each more times . Moreover, in these problems, practitioners may be risk-averse and hence prefer an input with both good average performance and small variability. To tackle both challenges, we propose the Batch Thompson Sampling for Replicable Experimental Design (BTS-RED) framework, which encompasses three algorithms. Our BTS-RED-Known and BTS-RED-Unknown algorithms, for, respectively, known and unknown noise variance, choose the number of replications adaptively rather than deterministically such that an input with a larger noise variance is replicated more times. As a result, despite the noise heteroscedasticity, both algorithms enjoy a theoretical guarantee and are asymptotically no-regret . Our Mean-Var-BTS-RED algorithm aims at risk-averse optimization and is also asymptotically no-regret. We also show the effectiveness of our algorithms in two practical real-world applications: precision agriculture and AutoML.

1 Introduction

Bayesian optimization (BO), which is a sequential algorithm for optimizing black-box and expensiveto-evaluate functions [14, 15], has found application in a wide range of experimental design problems [21]. Many such applications which use BO to accelerate the scientific discovery process [27] fall under the umbrella of AI for science ( AI4Science ). Many real-world experimental design problems, such as precision agriculture, share two inherent characteristics: (a) multiple experimental conditions are usually evaluated in parallel to take full advantage of the available experimental budget; (b) the evaluation of every experimental condition is usually replicated multiple times [31] because every experiment may be associated with a large and heteroscedastic (i.e., input-dependent) observation noise, in which case replication usually leads to better performances [2, 33, 43]. Replicating each evaluated experimental condition is also a natural choice in experimental design problems in which it incurs considerable setup costs to test every new experimental condition. This naturally induces an interesting challenge regarding the trade-off between input selection and replication: in every iteration of BO where we are given a fixed total experimental budget, should we evaluate more unique experimental conditions and replicate each of them fewer times or evaluate fewer unique conditions and replicate each more times ? Interestingly, this trade-off is also commonly found in

other applications such as automated machine learning (AutoML), in which parallel evaluations are often adopted to exploit all available resources [28] and heteroscedasticity is a prevalent issue [12]. Furthermore, these experimental design problems with large and heteroscedastic noise are often faced with another recurring challenge: instead of an input experimental condition (e.g., a hyperparameter configuration for an ML model) that produces a good performance (e.g., large validation accuracy) on average, some practitioners may be risk-averse and instead prefer an input that both yields a good average performance and has small variability. As a result, instead of only maximizing the mean of the black-box function, these risk-averse practitioners may instead look for inputs with both a large mean function value and a small noise variance [26, 33].

In this work, we provide solutions to both challenges in a principled way by proposing the framework of Batch Thompson Sampling for Replicable Experimental Design (BTS-RED). The first challenge regarding the trade-off between input selection and replication is tackled by the first two incarnations of our framework: the BTS-RED-Known (Sec. 3.1) and BTS-RED-Unknown (Sec. 3.2) algorithms, which are applicable to scenarios where the noise variance function is known or unknown, respectively. For batch selection, we adopt the Thompson sampling (TS) strategy because its inherent randomness makes it particularly simple to select a batch of inputs [28]. Moreover, previous works on BO have shown that the use of TS both allows for the derivation of theoretical guarantees [16, 28] and leads to strong empirical performances [16, 20]. For replication selection, instead of the common practice of replicating every queried input a fixed number of times, we choose the number of replications adaptively depending on the observation noise . Specifically, in every iteration, both algorithms repeat the following two steps until the total budget is exhausted: (a) choose an input query following the TS strategy, and then (b) adaptively choose the number of replications for the selected input such that an input with a larger noise variance is replicated more times . Of note, in spite of the noise heteroscedasticity, our principled approach to choosing the number of replications ensures that the effective noise variance R 2 of every queried input is the same (Sec. 3.1). This allows us to derive an upper bound on their cumulative regret and show that they are asymptotically noregret . Our theoretical guarantee formalizes the impact of the properties of the experiments, i.e., our regret upper bound becomes better if the total budget is increased or if the overall noise level is reduced. Importantly, our theoretical result provides a guideline on the choice of the effective noise variance parameter R 2 , which is achieved by minimizing the regret upper bound and allows R 2 to automatically adapt to the budgets and noise levels of different experiments (Sec. 3.1.2).

To handle the second challenge of risk-averse optimization, we propose the third variant of our BTS-RED framework named Mean-Var-BTS-RED (Sec. 4), which is a natural extension of BTSRED-Unknown. Mean-Var-BTS-RED aims to maximize the mean-variance objective function, which is a weighted combination of the mean objective function and negative noise variance function (Sec. 2). We prove an upper bound on the mean-variance cumulative regret of Mean-Var-BTS-RED (Sec. 4) and show that it is also asymptotically no-regret .

In addition to our theoretical contributions, we also demonstrate the practical efficacy of our algorithms in two real-world problems (Sec. 5). Firstly, in real-world precision agriculture experiments, plant biologists usually (a) evaluate multiple growing conditions in parallel, and (b) replicate each condition multiple times to get a reliable outcome [31]. Moreover, plant biologists often prefer more replicable conditions, i.e., inputs with small noise variances. This is hence an ideal application for our algorithms. So, we conduct an experiment using real-world data on plant growths, to show the effectiveness of our algorithms in precision agriculture (Sec. 5.2). Next, we also apply our algorithms to AutoML to find hyperparameter configurations with competitive and reproducible results across different AutoML tasks (Sec. 5.3). The efficacy of our algorithms demonstrates their capability to improve the reproducibility of AutoML tasks which is an important issue in AutoML [25].

2 Background

We denote by f : X → R the objective function we wish to maximize, and by σ 2 : X → R + the input-dependent noise variance function. We denote the minimum and maximum noise variance as σ 2 min and σ 2 max . For simplicity, we assume that the domain X is finite, since extension to compact domains can be easily achieved via suitable discretizations [5]. After querying an input x ∈ X , we observe a noisy output y = f ( x ) + ϵ where ϵ ∼ N (0 , σ 2 ( x )) . In every iteration t , we select a batch of b t ≥ 1 inputs { x ( b ) t } b =1 ,...,b t , and query every x ( b ) t with n ( b ) t ≥ 1 parallel processes. We denote the total budget as B such that ∑ b t b =1 n ( b ) t ≤ B , ∀ t ≥ 1 . We model the function f

using a Gaussian process (GP) [40]: GP ( µ ( · ) , k ( · , · )) , where µ ( · ) is a mean function which we assume w.l.o.g. µ ( x ) = 0 and k ( · , · ) is a kernel function for which we focus on the commonly used squared exponential (SE) kernel. In iteration t , we use the observation history in the first t -1 iterations (batches) to calculate the GP posterior GP ( µ t -1 ( · ) , σ 2 t -1 ( · , · )) , in which µ t -1 ( · ) and σ 2 t -1 ( · , · ) represent the GP posterior mean and covariance functions (details in Appendix A). For BTS-RED-Unknown and Mean-Var-BTS-RED (i.e., when σ 2 ( · ) is unknown), we use another GP, denoted as GP ′ , to model -σ 2 ( · ) (Sec. 3.2), and denote its posterior as GP ′ ( µ ′ t -1 ( · ) , σ ′ 2 t -1 ( · , · )) .

In our theoretical analysis of BTS-RED-Known and BTS-RED-Unknown where we aim to maximize f , we follow previous works on batch BO [11, 18, 35] and derive an upper bound on the batch cumulative regret R T = ∑ T t =1 min b ∈ [ b t ] [ f ( x ∗ ) -f ( x ( b ) t )] , in which x ∗ ∈ arg max x ∈X f ( x ) and we have used [ b t ] to denote { 1 , . . . , b t } . We show (Sec. 3) that both BTS-RED-Known and BTS-REDUnknown enjoy a sub-linear upper bound on R T , which suggests that as T increases, a global optimum x ∗ is guaranteed to be queried since the batch simple regret S T = min t ∈ [ T ] min b ∈ [ b t ] [ f ( x ∗ ) -f ( x ( b ) t )] ≤ R T /T goes to 0 asymptotically. We analyze the batch cumulative regret because it allows us to show the benefit of batch evaluations, and our analysis can also be modified to give an upper on the sequential cumulative regret of R ′ T = ∑ T t =1 ∑ b t b =1 [ f ( x ∗ ) -f ( x ( b ) t )] (Appendix D). Our Mean-Var-BTS-RED aims to maximize the mean-variance objective function h ω ( x ) = ωf ( x ) -(1 -ω ) σ 2 ( x ) = ωf ( x ) + (1 -ω ) g ( x ) , in which we have defined g ( x ) ≜ -σ 2 ( x ) , ∀ x ∈ X . The user-specified weight parameter ω ∈ [0 , 1] reflects our relative preference for larger mean function values or smaller noise variances. Define the mean-variance batch cumulative regret as R MV T = ∑ T t =1 min b ∈ [ b t ] [ h ω ( x ∗ ω ) -h ω ( x ( b ) t )] where x ∗ ω ∈ arg max x ∈X h ω ( x ) . We also prove a sub-linear upper bound on R MV T for Mean-Var-BTS-RED (Sec. 4).

3 BTS-RED-Known and BTS-RED-Unknown

Here, we firstly introduce BTS-RED-Known and its theoretical guarantees (Sec. 3.1), and then discuss how it can be extended to derive BTS-RED-Unknown (Sec. 3.2).

3.1 BTS-RED with Known Noise Variance Function

3.1.1 BTS-RED-Known

Algorithm 1 BTS-RED-Known.

1: for t = 1 , 2 , . . . , T do

2: b = 0 , n (0) t = 0

4: b ← b +1

3: while ∑ b b ′ =0 n ( b ′ ) t < B do

5: Sample a function f ( b ) t from the GP posterior of GP ( µ t -1 ( · ) , β 2 t σ 2 t -1 ( · , · )) (Sec. 2)

6: Choose x ( b ) t = arg max x ∈X f ( b ) t ( x ) and n ( b ) t = ⌈ σ 2 ( x ( b ) t ) /R 2 ⌉

7: b t = b -1

8: for b ∈ [ b t ] , query x ( b ) t with n ( b ) t parallel processes

9: for b ∈ [ b t ] , observe { y ( b ) t,n } n ∈ [ n ( b ) t ] . Calculate their empirical mean y ( b ) t = (1 /n ( b ) t ) ∑ n ( b ) t n =1 y ( b ) t,n

10: Use { ( x ( b ) t , y ( b ) t ) } b ∈ [ b t ] to update posterior of GP

BTS-RED-Known (Algo. 1) assumes that σ 2 ( · ) is known. In every iteration t , to sequentially select every x ( b ) t and its corresponding n ( b ) t , we repeat the following process until the total number of replications has consumed the total budget B (i.e., until ∑ b b ′ =1 n ( b ′ ) t ≥ B , line 3 of Algo. 1):

line 5 : sample a function f ( b ) t from GP ( µ t -1 ( · ) , β 2 t σ 2 t -1 ( · , · )) ( β t will be defined in Theorem 3.1);

line 6 : choose x ( b ) t = arg max x ∈X f ( b ) t ( x ) by maximizing the sampled function f ( b ) t , and choose n ( b ) = ⌈ σ 2 ( x ( b ) ) /R 2 ⌉ , where ⌈·⌉ is the ceiling operator and R 2 is the effective noise variance .

After the entire batch of b t inputs have been selected, every x ( b ) t is queried with n ( b ) t parallel processes (line 8), and the empirical mean y ( b ) of these n ( b ) observations is calculated (line 9).

t t t t

Finally, { ( x ( b ) t , y ( b ) t ) } b ∈ [ b t ] are used to update the posterior of GP (line 10). Of note, since the observation noise is assumed to be Gaussian-distributed with a variance of σ 2 ( x ( b ) t ) (Sec. 2), after querying x ( b ) t independently for n ( b ) t times, the empirical mean y ( b ) t follows a Gaussian distribution with noise variance σ 2 ( x ( b ) t ) /n ( b ) t . Next, since we select n ( b ) t by n ( b ) t = ⌈ σ 2 ( x ( b ) t ) /R 2 ⌉ (line 6), σ 2 ( x ( b ) t ) /n ( b ) t is guaranteed to be upper-bounded by R 2 . In other words, every observed empirical mean y ( b ) t follows a Gaussian distribution with a noise variance that is upper-bounded by the effective noise variance R 2 . This is crucial for our theoretical analysis since it ensures that the effective noise variance is R -sub-Gaussian and thus preserves the validity of the GP-based confidence bound [8].

In practice, since our BTS-RED-Known algorithm only aims to maximize the objective function f (i.e., we are not concerned about learning the noise variance function), some replications may be wasted on undesirable input queries (i.e., those with small values of f ( x ) ) especially in the initial stage when our algorithm favours exploration. To take this into account, we adopt a simple heuristic: we impose a maximum number of replications denoted as n max , and set n max = B / 2 in the first T/ 2 iterations and n max = B afterwards. This corresponds to favouring exploration of more inputs (each with less replications) initially and preferring exploitation in later stages. This technique is also used for BTS-RED-Unknown (Sec. 3.2) yet not adopted for Mean-Var-BTS-RED (Sec. 4) since in mean-variance optimization, we also aim to learn (and minimize) the noise variance function.

Due to our stopping criterion for batch selection (line 3 ), in practice, some budgets may be unused in an iteration. E.g., when B = 50 , if ∑ b -1 b ′ =1 n ( b ′ ) t = 43 after the first b -1 selected queries and the newly selected n t for the b th query x ( b ) t is n ( b ) t = 12 , then the termination criterion is met (i.e., ∑ b b ′ =1 n ( b ′ ) t ≥ B ) and only 43 / 50 of the budgets are used. So, we adopt a simple technique: in the example above, we firstly evaluate the last selected x ( b ) t for 7 times, and in the next iteration t +1 , we start by completing the unfinished evaluation of x ( b ) t by allocating 12 -7 = 5 replications to x ( b ) t . Next, we run iteration t +1 with the remaining budget, i.e., we let B = 50 -5 = 45 in iteration t +1 .

3.1.2 Theoretical Analysis of BTS-RED-Known

Following the common practice in BO [8], we assume f lies in a reproducing kernel Hilbert space (RKHS) induced by an SE kernel k : ∥ f ∥ H k ≤ B for some B > 0 where ∥·∥ H k denotes the RKHS norm. Theorem 3.1 below gives a regret upper bound of BTS-RED-Known (proof in Appendix B).

Theorem 3.1 (BTS-RED-Known) . Choose δ ∈ (0 , 1) . Define τ t -1 ≜ ∑ t -1 t ′ =1 b t ′ , and define β t ≜ B + R √ 2(Γ τ t -1 +1+log(2 /δ )) where Γ τ t -1 denotes the maximum information gain about f from any τ t -1 observations. With probability of at least 1 -δ ( ˜ O ignores all log factors),

C is a constant s.t. max A ⊂X , | A |≤ B I ( f ; y A | y 1: t -1 ) ≤ C, ∀ t ≥ 1 . I ( f ; y A | y 1: t -1 ) is the information gain from observations y A at inputs A , given observations y 1: t -1 in the first t -1 iterations.

It has been shown by [19] that by running uncertainty sampling (i.e., choosing the initial inputs by sequentially maximizing the GP posterior variance) as the initialization phase for a finite number (independent of T ) of iterations, C can be chosen to be a constant independent of B and T . As a result, the regret upper bound from Theorem 3.1 can be simplified into R T = ˜ O ( √ R 2 / ( B / ⌈ σ 2 max R 2 ⌉ -1) √ T Γ T B (1 + √ Γ T B ) ) . Therefore, for the SE kernel for which Γ T B = O (log d +1 ( T B )) , our regret upper bound is sub-linear, which indicates that our BTS-RED-Known is asymptotically no-regret . Moreover, the benefit of a larger total budget B is also reflected from our regret upper bound since it depends on the total budget B via ˜ O ((log d +1 ( T B ) + log ( d +1) / 2 ( T B )) / √ B ) , which is decreasing as the total budget B increases. In addition, the regret upper bound is decreased if σ 2 max becomes smaller, which implies that the performance of our algorithm is improved if the overall noise level is reduced. Therefore, Theorem 3.1 formalizes the impacts of the experimental properties (i.e., the total budget and the overall noise level) on the performance of BTS-RED-Known.

Homoscedastic Noise. In the special case of homoscedastic noise, i.e., σ 2 ( x ) = σ 2 const , ∀ x ∈ X , then n t = ⌈ σ 2 const /R 2 ⌉ ≜ n const and b t = ⌊ B /n const ⌋ ≜ b 0 , ∀ t ∈ [ T ] . That is, our algorithm reduces

to standard (synchronous) batch TS proposed in [28] where the batch size is b 0 and every query is replicated n const times. In this case, the regret upper bound becomes: R T = ˜ O ( R 1 √ b 0 √ T Γ Tb 0 (1 + √ Γ Tb 0 )) (Appendix B.1).

Theoretical Guideline on the Choice of R 2 . Theorem 3.1 also provides an interesting insight on the choice of the effective noise variance R 2 . In particular, our regret upper bound depends on R 2 through the term √ R 2 / [ B / ( σ 2 max /R 2 +1) -1] . 1 By taking the derivative of this term w.r.t. R 2 , we have shown (Appendix C) that the value of R 2 that minimizes this term is obtained at R 2 = σ 2 max ( √ B +1) / ( B -1) . In other words, R 2 should be chosen as a fraction of σ 2 max (assuming B > 4 s.t. ( √ B +1) / ( B -1) < 1 ). In this case, increasing the total budget B naturally encourages more replications. Specifically, increasing B reduces ( √ B +1) / ( B -1) and hence decreases the value of R 2 , which consequently encourages the use of larger n t 's (line 6 of Algo. 1) and allows every selected input to be replicated more times. For example, when the total budget is B = 16 , R 2 should be chosen as R 2 = σ 2 max / 3 ; when B = 100 , then we have R 2 = σ 2 max / 9 . We will follow this theory-inspired choice of R 2 in our experiments in Sec. 5 (with slight modifications).

Improvement over Uniform Sample Allocation. For the naive baseline of uniform sample allocation (i.e., replicating every input a fixed number n 0 ≤ B of times), the resulting effective observation noise would be ( σ max / √ n 0 ) -sub-Gaussian. This would result in a regret upper bound which can be obtained by replacing the term √ R 2 / ( B / ⌈ σ 2 max R 2 ⌉ -1) (Theorem 3.1) by σ max / √ n 0 (for simplicity, we have ignored the non-integer conditions, i.e., the ceiling operators). Also note that with our optimal choice of R 2 (the paragraph above), it can be easily verified that the term √ R 2 / ( B / ⌈ σ 2 max R 2 ⌉ -1) (Theorem 3.1) can be simplified to σ max / √ B . Therefore, given that n 0 ≤ B , our regret upper bound (with the scaling of σ max / √ B ) is guaranteed to be no worse than that of uniform sample allocation (with the scaling of σ max / √ n 0 ).

3.2 BTS-RED with Unknown Noise Variance Function

Here we consider the more common scenario where the noise variance function σ 2 ( · ) is unknown by extending BTS-RED-Known while preserving its theoretical guarantee.

3.2.1 Modeling of Noise Variance Function

We use a separate GP (denoted as GP ′ ) to model the negative noise variance function g ( · ) = -σ 2 ( · ) and use it to build a high-probability upper bound U σ 2 t ( · ) on the noise variance function σ 2 ( · ) . 2 After this, we can modify the criteria for selecting n ( b ) t (i.e., line 6 of Algo. 1) to be n ( b ) t = ⌈ U σ 2 t ( x ( b ) t ) /R 2 ⌉ , which ensures that U σ 2 t ( x ( b ) t ) /n ( b ) t ≤ R 2 . As a result, the condition of σ 2 ( x ( b ) t ) /n ( b ) t ≤ R 2 is still satisfied (with high probability), which implies that the observed empirical mean at every queried x ( b ) t is still R -sub-Gaussian (Sec. 3.1.1) and theoretical guarantee of Theorem 3.1 is preserved. To construct GP ′ , we use the (negated) unbiased empirical noise variance ˜ y ( b ) t as the noisy observation:

where g ( x ( b ) t ) = -σ 2 ( x ( b ) t ) is the negative noise variance at x ( b ) t , and ϵ ′ is the noise. In BTS-REDUnknown, we use pairs of { ( x ( b ) t , ˜ y ( b ) t ) } to update the posterior of GP ′ . We impose a minimum number of replications n min ≥ 2 for every queried input to ensure reliable estimations of ˜ y ( b ) t .

3.2.2 Upper Bound on Noise Variance Function

Assumptions. Similar to Theorem 3.1, we assume that g lies in an RKHS associated with an SE kernel k ′ : ∥ g ∥ H k ′ ≤ B ′ for some B ′ > 0 , which intuitively assumes that the (negative) noise variance

varies smoothly across the domain X . We also assume that the noise ϵ ′ is R ′ -sub-Gaussian and justify this below by showing that ϵ ′ is bounded (with high probability).

ϵ ′ is R ′ -sub-Gaussian. Since the empirical variance of a Gaussian distribution (1) follows a Chisquared distribution, we can use the concentration of Chi-squared distributions to show that with probability of ≥ 1 -α , ϵ ′ is bounded within [ L α , U α ] , where L α = σ 2 min ( χ 2 n min -1 ,α/ 2 / ( n min -1) -1) , U α = σ 2 max ( χ 2 n min -1 , 1 -α/ 2 / ( n min -1) -1) . Here χ 2 n min -1 ,η denotes η th -quantile of the Chi-squared distribution with n min -1 degrees of freedom ( η = α/ 2 or 1 -α/ 2 ). By choosing α = δ/ (4 T B ) ( δ is from Theorem 3.1), we can ensure that with probability of ≥ 1 -δ/ 4 , ϵ ′ is bounded within [ L α , U α ] for all x [ b ] t . In other words, with probability of ≥ 1 -δ/ 4 , the noise ϵ ′ in (1) is zero-mean and bounded within [ L α , U α ] , which indicates that ϵ ′ is R ′ -sub-Gaussian with R ′ = ( U α -L α ) / 2 . More details are given in Appendix E. Note that the value of R ′ derived here is expected to be overly pessimistic, so, we expect smaller values of R ′ to be applicable in practice.

Upper Bound Construction. With the assumptions of ∥ g ∥ H k ′ ≤ B ′ and ϵ ′ is R ′ -sub-Gaussian, we can construct the upper bound U σ 2 t ( · ) . Denote by Γ ′ τ t -1 the maximum information gain about g from any τ t -1 = ∑ t -1 t ′ =1 b t ′ observations, define β ′ t ≜ B ′ + R ′ √ 2(Γ ′ τ t -1 +1+log(4 /δ )) , and represent the GP posterior mean and standard deviation for GP ′ as µ ′ t -1 ( · ) and σ ′ t -1 ( · ) . Then we have that

with probability of ≥ 1 -δ/ 2 . The error probabilities come from applying Theorem 2 of [8] ( δ/ 4 ) and assuming that ϵ ′ is R ′ -sub-Gaussian ( δ/ 4 ). This implies that -σ 2 ( x ) = g ( x ) ≥ µ ′ t -1 ( x ) -β ′ t σ ′ t -1 ( x ) , and hence σ 2 ( x ) ≤ -µ ′ t -1 ( x )+ β ′ t σ ′ t -1 ( x ) , ∀ x ∈ X , t ∈ [ T ] . Therefore, we can choose the upper bound on the noise variance (Sec. 3.2.1) as U σ 2 t ( x ) = -µ ′ t -1 ( x ) + β ′ t σ ′ t -1 ( x ) .

BTS-RED-Unknown Algorithm. To summarize, we can obtain BTS-RED-Unknown (Algo. 3, Appendix F) by modifying the selection criterion of n t (line 6 of Algo. 1) to be n ( b ) t = ⌈ ( -µ ′ t -1 ( x ( b ) t ) + β ′ t σ ′ t -1 ( x ( b ) t )) /R 2 ⌉ . As a result, BTS-RED-Unknown enjoys the same regret upper bound as Theorem 3.1 (after replacing δ in Theorem 3.1 by δ/ 2 ). Intuitively, using an upper bound U σ 2 t ( x ( b ) t ) in the selection of n t implies that if we are uncertain about the noise variance at some input location x ( b ) t (i.e., if σ ′ t -1 ( x ( b ) t ) is large), we choose to be conservative and use a large number of replications n t .

4 Mean-Var-BTS-RED

We extend BTS-RED-Unknown (Sec. 3.2) to maximize the mean-variance objective function: h ω ( x ) = ωf ( x ) -(1 -ω ) σ 2 ( x ) , to introduce Mean-Var-BTS-RED (Algo. 2). In contrast to BTS-RED-Unknown, Mean-Var-BTS-RED chooses every input query x ( b ) t by maximizing the weighted combination of two functions sampled from, respectively, the posteriors of GP and GP ′ (lines 5-6 of Algo. 2), while n t is chosen (line 7 of Algo. 2) in the same way as BTS-RED-Unknown. This naturally induces a preference for inputs with both large values of f and small values of σ 2 ( · ) , and hence allows us to derive an upper bound on R MV T (proof in Appendix G):

Theorem 4.1 (Mean-Var-BTS-RED) . With probability of at least 1 -δ ,

C is a constant s.t. max A ⊂X , | A |≤ B I ( f ; y A | y 1: t -1 ) ≤ C , max A ⊂X , | A |≤ B I ( g ; ˜ y A | ˜ y 1: t -1 ) ≤ C .

Note that Γ T B and Γ ′ T B may differ since the SE kernels k and k ′ , which are used to model f and g respectively, may be different. Similar to Theorem 3.1, if we run uncertainty sampling for a finite number (independent of T ) of initial iterations using either k or k ′ (depending on whose lengthscale is smaller), then C can be chosen to be a constant independent of B and T . Refer to Lemma G.6 (Appendix G) for more details. As a result, the regret upper bound in Theorem 4.1 is also sub-linear since both k and k ′ are SE kernels and hence Γ T B = O (log d +1 ( T )) and Γ ′ T B = O (log d +1 ( T )) . The regret upper bound can be viewed as a weighted combination of the regrets associated with f and g . Intuitively, if ω is larger (i.e., if we place more emphasis on maximizing f than g ), then a larger proportion of the regrets is incurred due to our attempt to maximize the function f .

5 Experiments

For BTS-RED-Known and BTS-RED-Unknown which only aim to maximize the objective function f , we set n max = B / 2 in the first T/ 2 iterations and n max = B subsequently (see Sec. 3.1.1 for more details), and set n max = B in all iterations for Mean-Var-BTS-RED. We set n min = 2 unless specified otherwise, however, it is recommended to make n min larger in experiments where the overall noise variance is large (e.g., we let n min = 5 in Sec. 5.2). We use random search to select the initial inputs instead of the uncertainty sampling initialization method indicated by our theoretical results (Sec. 3.1.2) because previous work [28] and our empirical results show that they lead to similar performances (Fig. 8 in App. H.1). We choose the effective noise variance R 2 by following our theoretical guideline in Sec. 3.1.2, i.e., R 2 = σ 2 max ( √ B +1) / ( B -1) which minimizes the regret upper bound in Theorem 3.1. 3 However, in practice, this choice may not be optimal because we derived it by minimizing an upper bound which is potentially loose (e.g., we have ignored all log factors). So, we introduce a tunable parameter κ > 0 and choose R 2 as R 2 = κσ 2 max ( √ B +1) / ( B -1) . As a result, we both enjoy the flexibility of tuning our preference for the overall number of replications (i.e., a smaller κ leads to larger n t 's in general) and preserve the ability to automatically adapt to the total budget (via B ) and the overall noise level (via σ 2 max ). When the noise variance is unknown (i.e., σ 2 max is unknown), we approximate σ 2 max by the maximum observed empirical noise variance and update our approximation after every iteration. To demonstrate the robustness of our methods, we only use two values of κ = 0 . 2 and κ = 0 . 3 in all experiments. Of note, our methods with κ = 0 . 3 perform the best in almost all experiments (i.e., green curves in all figures), and κ = 0 . 2 also consistently performs well.

Following the common practice of BO [11, 20, 28, 32, 35], we plot the (batch) simple regret or the best observed function value up to an iteration. In all experiments, we compare with the most natural baseline of batch TS with a fixed number of replications. For mean optimization problems (i.e., maximize f ), we also compare with standard sequential BO algorithms such as GP-UCB and GP-TS, but they are significantly outperformed by both our algorithms and batch TS which are able to exploit batch evaluations (Secs. 5.1 and 5.2). Therefore, we do not expect existing sequential algorithms to achieve comparable performances to our algorithms due to their inability to exploit batch evaluations. For mean-variance optimization, we additionally compare with the recently introduced Risk-Averse Heteroscedastic BO (RAHBO) [33] (Sec. 6), which is the state-of-the-art method for risk-averse BO with replications. Some experimental details are postponed to Appendix H.

5.1 Synthetic Experiments

We sample two functions from two different GPs with the SE kernel (defined on a discrete 1-D domain within [0 , 1] ) and use them as f ( · ) and σ 2 ( · ) , respectively. We use B = 50 .

Mean Optimization. The mean and noise variance functions used here are visualized in Fig. 1a. This synthetic experiment is used to simulate real-world scenarios where practitioners are risk-neutral and hence only aim to select an input with a large mean function value. After every iteration (batch) t , an

0.0

0.2

0.4

0.6

0.8

1.0

input

0.0

0.5

1.0

0.0

0.2

0.4

0.6

0.8

1.0

input

0.0

0.1

0.2

0.3

0.4

0.5

0.6

noise var function

nt

/50 for

κ

=0.2

nt

/50 for

κ

=0.3

0

20

40

60

Iterations

0.00

0.05

0.10

0.15

Simple Regret

Batch TS (

nt

=1)

Batch TS (

nt

=5)

Batch TS (

nt

=10)

Batch TS (

nt

=20)

BTS-RED-Known (

κ

=0.3)

BTS-RED-Known (

κ

=0.2)

BTS-RED-Unknown (

κ

=0.3)

BTS-RED-Unknown (

κ

=0.2)

0

20

40

60

Iterations

0.00

0.01

0.02

0.03

0.04

0.05

(Mean-Var) Simple Regret

Batch TS (

nt

=5)

Batch TS (

nt

=10)

Batch TS (

nt

=20)

κ

=0.3)

κ

=0.2)

RAHBO

(a)

(b)

(c)

(d)

algorithm reports the selected input with the largest empirical mean from its observation history, and we evaluate the simple regret at iteration t as the difference between the objective function values at the global maximum x ∗ and at the reported input. To demonstrate the consistency of our performance, we also tested an alternative reporting criteria which reports the input with the larger LCB value in every iteration, and the results (Fig. 7b in Appendix H.1) are consistent with our main results (Fig. 1c). Fig. 1b plots the average n t (vertical axis) chosen by BTS-RED-Unknown for every queried input (horizontal axis), which shows that larger n t 's are selected for inputs with larger noise variances in general and that a smaller κ = 0 . 2 indeed increases our preference for larger n t 's.

The results (simple regrets) are shown in Fig. 1c. As can be seen from the figure, for Batch TS with a fixed n t , smaller values of n t such as n t = 5 usually lead to faster convergence initially due to the ability to quickly explore more unique inputs, however, their performances deteriorate significantly in the long run due to inaccurate estimations; in contrast, larger n t 's such as n t = 20 result in slower convergence initially yet lead to better performances (than small fixed n t 's) in later stages. Of note, Batch TS with n t = 1 (gray curve) represents standard batch TS ( B = 50 ) without replications [28], which underperforms significantly and hence highlights the importance of replications in experiments with large noise variance. Moreover, our BTS-RED-Known and BTS-RED-Unknown (especially with κ = 0 . 3 ) consistently outperform Batch TS with fixed n t . We also demonstrate our robustness against κ in this experiment by showing that our performances are consistent for a wide range of κ 's (Fig. 7a in App. H.1). In addition, we show that sequential BO algorithms (i.e., GP-TS, GP-UCB, and GP-UCB with heteroscedastic GP) which cannot exploit batch evaluations fail to achieve comparable performances to batch TS, BTS-RED and BTS-RED-Unknown (Fig. 5 in App. H.1).

Mean-variance Optimization. Here we evaluate our Mean-Var-BTS-RED. We simulate this scenario with the synthetic function in Fig. 6a (App. H.1), for which the global maximums of the mean and mean-variance ( ω = 0 . 3 ) objective functions are different (Fig. 6b). After every iteration (batch) t , we report the selected input with the largest empirical mean-variance value (i.e., weighted combination of the empirical mean and variance), and evaluate the mean-variance simple regret at iteration t as the difference between the values of the mean-variance objective function h ω at the the global maximum x ∗ ω and at the reported input. The results (Fig. 1d) show that our Mean-Var-BTS-RED (again especially with κ = 0 . 3 ) outperforms other baselines. Since RAHBO is sequential and uses a fixed number of replications, we use B = 50 replications for every query for a fair comparison. RAHBO underperforms here which is likely due to its inability to leverage batch evaluations.

5.2 Real-world Experiments on Precision Agriculture

Plant biologists often need to optimize the growing conditions of plants (e.g., the amount of different nutrients) to increase their yield. The common practice of manually tuning one nutrient at a time is considerably inefficient and hence calls for the use of the sample-efficient method of BO. Unfortunately, plant growths are usually (a) time-consuming and (b) associated with large and heteroscedastic noise. So, according to plant biologists, in real lab experiments, (a) multiple growing conditions are usually tested in parallel and (b) every condition is replicated multiple times to get a reliable outcome [31]. This naturally induces a trade-off between evaluating more unique growing conditions vs. replicating every condition more times, and is hence an ideal application for our algorithms. We tune the pH value (in [2 . 5 , 6 . 5] ) and ammonium concentration (denoted as NH3, in [0 , 30000] uM). in order to maximize the leaf area and minimize the tipburn area after harvest. We perform real lab experiments using the input conditions from a regular grid within the 2-D domain, and then use the collected data to learn two separate heteroscedastic GPs for, respectively, leaf area and tipburn area. Each learned GP can output the predicted mean and variance (for leaf area or tipburn area) at every input in the 2-D domain, and can hence be used as the groundtruth mean f ( · ) and noise variance σ 2 ( · ) functions. We perform two sets of experiments, with the goal of maximizing (a) the

0

20

40

60

80

100

Iterations

134

136

138

140

142

Leaf Area

Batch TS (

nt

=5)

Batch TS (

nt

=10)

Batch TS (

nt

=20)

BTS-RED-Unknown (

κ

=0.3)

BTS-RED-Unknown (

κ

=0.2)

0

20

40

60

80

100

Iterations

65

70

75

80

85

Leaf Area (Mean-Var)

Mean-Var Batch TS (

nt

=10)

Mean-Var Batch TS (

nt

=20)

Mean-Var Batch TS (

nt

=25)

κ

=0.3)

κ

=0.2)

RAHBO

0

20

40

60

80

100

Iterations

106

108

110

112

Leaf Area - Tipburn Area

Batch TS (

nt

=5)

Batch TS (

nt

=10)

Batch TS (

nt

=20)

BTS-RED-Unknown (

κ

=0.3)

BTS-RED-Unknown (

κ

=0.2)

0

20

40

60

80

100

Iterations

50

55

60

65

Leaf Area - Tipburn Area (Mean-Var)

Mean-Var Batch TS (

nt

=5)

Mean-Var Batch TS (

nt

=10)

Mean-Var Batch TS (

nt

=20)

κ

=0.3)

κ

=0.2)

RAHBO

(a)

(b)

(c)

(d)

2.5

3.3

4.1

4.9

5.7

6.5

pH

0

6

12

18

24

30

NH3 (×1000)

7

7

22

9

7

14

7

8

11

7

5

11

10

8

11

5

5 5

14

8

12

16

10

11

12

9

5

6

6

11

6

10

6

10

7

9

5

10

6

7

5

6

8

10

6

6

6

13

13

7

15

6

12

13

16

16

7

15

10

11

5

12

8

8

5

5

10

5

9

7

9

11

12

15

14

5

7

11

8

8

7

5

14

9

7

6

11

10

5

16

10

11

7

14

10

6

7

10

5

10

11

5

13

6

12

15

7

12

6

5

6

11

5

10 12

5

9

6

9

11

9

5

7

8

5

5

6

5

10

9

6

8

6

5

9

7

6

5

10

7

5

Mean

20

40

60

80

100

120

140

2.5

3.3

4.1

4.9

5.7

6.5

pH

0

6

12

18

24

30

NH3 (×1000)

7

7

22

9

7

14

7

8

11

7

5

11

10

8

11

5

5 5

14

8

12

16

10

11

12

9

5

6

6

11

6

10

6

10

7

9

5

10

6

7

5

6

8

10

6

6

6

13

13

7

15

6

12

13

16

16

7

15

10

11

5

12

8

8

5

5

10

5

9

7

9

11

12

15

14

5

7

11

8

8

7

5

14

9

7

6

11

10

5

16

10

11

7

14

10

6

7

10

5

10

11

5

13

6

12

15

7

12

6

5

6

11

5

10 12

5

9

6

9

11

9

5

7

8

5

5

6

5

10

9

6

8

6

5

9

7

6

5

10

7

5

3

4

5

6

7

8

9

0

20

40

60

Iterations

0.00

0.02

0.04

0.06

0.08

0.10

Simple Regret

Batch TS (fixed

nt

=5)

Batch TS (fixed

nt

=10)

Batch TS (fixed

nt

=20)

BTS-RED-Unknown (

κ

=0.3)

BTS-RED-Unknown (

κ

=0.2)

0

20

40

60

Iterations

0.00

0.01

0.02

0.03

(Mean-Var) Simple Regret

nt

=5)

nt

=10)

nt

=20)

κ

=0.3)

κ

=0.2)

(a)

(b)

(c)

(d)

leaf area and (b) a weighted combination of the leaf area ( × 0 . 8 ) and negative tipburn area ( × 0 . 2 ). For both experiments, we run BTS-RED-Unknown and Mean-Var-BTS-RED to maximize the mean and mean-variance objectives ( ω = 0 . 975 ), respectively. We set B = 50 , n min = 5 and n max = 50 .

Fig. 2 shows the results for maximizing the leaf area (a,b) and weighted combination of leaf area and negative tipburn area (c,d). Our BTS-RED-Unknown and Mean-Var-BTS-RED with κ = 0 . 2 and κ = 0 . 3 consistently outperform Batch TS, as well as RAHBO in Figs. 2b and d. For mean optimization, we also compare with sequential BO methods (Fig. 10 in Appendix H.2), which again are unable to perform comparably with other algorithms that exploit batch evaluations. Figs. 3a and b visualize the groundtruth mean and noise variance functions for the leaf area, including the locations of some queried inputs (the selected inputs after every 4 iterations) and their corresponding n t 's. Similarly, Figs. 9a and b (Appendix H.2) show the queried inputs and the n t 's of Mean-Var-BTS-RED ( ω = 0 . 975 ), illustrated on heat maps of the mean-variance objective (a) and noise variance functions (b). These figures demonstrate that most of our input queries fall into regions with large (either mean or mean-variance) objective function values (Figs. 3a and 9a) and that n t is in general larger at those input locations with larger noise variance (Figs. 3b and 9b). We have included GIF animations for Figs. 3 and 9 in the supplementary material. Our results here showcase the capability of our algorithms to improve the efficiency of real-world experimental design problems.

5.3 Real-World Experiments on AutoML

Reproducibility is an important desiderata in AutoML problems such as hyperparameter tuning [25], because the performance of a hyperparameter configuration may vary due to a number of factors such as different datasets, parameter initializations, etc. For example, some practitioners may prefer hyperparameter configurations that consistently produce well-performing ML models for different datasets. We adopt the EMNIST dataset which is widely used in multi-task learning [10, 15]. EMNIST consists of images of hand-written characters from different individuals, and each individual corresponds to a separate image classification task . Here we tune two SVM hyperparameters: the penalty and RBF kernel parameters, both within [0 . 0001 , 2] . We firstly construct a uniform 2-D grid of the two hyeprparameters and then evaluate every input on the grid using 100 tasks (i.e., image classification for 100 different individuals) to record the observed mean and variance as the groundtruth mean and variance. Refer to Figs. 11a, b and c (Appendix H.3) for the constructed mean, variance and mean-variance ( ω = 0 . 2 ) functions. Fig. 3c and d plot the results ( B = 50 ) for mean (c) and mean-variance (d) optimization. Our BTS-RED-Unknown and Mean-Var-BTS-RED with both κ = 0 . 2 and 0 . 3 perform competitively (again especially κ = 0 . 3 ), which shows their potential to improve the efficiency and reproducibility of AutoML. RAHBO underperforms significantly (hence omitted from Fig. 3d), which is likely due to the small noise variance (Fig. 11b) which favors methods with small n t 's. Specifically, methods with small n t 's can obtain reliable estimations (due to small

noise variance) while enjoying the advantage of evaluating a large number b t of unique inputs in every iteration. This makes RAHBO unfavorable since it is a sequential algorithm with b t = 1 .

Experiments Using Different Budgets B . Here we test the performances of our algorithms with different budgets (i.e., different from the B = 50 used in the main experiments above) using the AutoML experiment. The results (Fig. 12 in App. H.3) show that the performance advantages of our algorithms (again especially with κ = 0 . 3 ) are still consistent with a larger or smaller budget.

Additional Experiments with Higher-Dimensional Inputs. To further verify the practicality of our proposed algorithms, here we adopt two additional experiments with higher-dimensional continuous input domains. Specifically, we tune d = 12 and d = 14 parameters of a controller for a Lunar-Lander task and a robot pushing task, respectively, and both experiments have widely used by previous works on high-dimensional BO [16, 20] (more details in App. H.4). In both experiments, the heteroscedastic noises arise from random environemntal factors. The results (Fig. 13 in App. H.4) show that our algorithms, again especially with κ = 0 . 3 , still consistently achieve compelling performances.

6 Related Works

BO has been extended to the batch setting in recent years [9, 11, 19, 22, 38, 44, 47]. The work of [28] proposed a simple batch TS method by exploiting the inherent randomness of TS. Interestingly, as we discussed in Sec. 3.1.2, the method from [28] is equivalent to a reduced version of our BTSRED-Known with homoscedastic noise, and our Theorem 3.1 provides a theoretical guarantee on its frequentist regret (in contrast to the Bayesian regret analyzed in [28]). The work of [2] aimed to adaptively choose whether to explore a new query or to replicate a previous query. However, their method requires additional heuristic techniques to achieve replications and hence has no theoretical guarantees, in stark contrast to our simple and principled way for replication selection (Sec. 3). Moreover, their method does not support batch evaluations, and is unable to tackle riskaverse optimization. Recently, [43] proposed to select a batch of queries while balancing exploring new queries and replicating existing ones. However, unlike our simple and principled algorithms, their method requires complicated heuristic procedures for query/replication selection and batch construction, and hence does not have theoretical guarantees. Moreover, their method also only focuses on standard mean optimization and cannot be easily extended for risk-averse optimization.

The work of [23] used a heteroscedastic GP [29] as the surrogate model for risk-averse optimization. The works of [6, 26, 36, 37, 42] considered risk-averse BO, however, these works require the ability to observe and select an environmental variable, which is usually either not explicitly defined or uncontrollable in practice (e.g., our experiments in Sec. 5). The recent work of [33] modified BO to maximize the mean-variance objective and derived theoretical guarantees using results from [30]. Their method uses a heteroscedastic GP as the surrogate model and employs another (homoscedastic) GP to model the observation noise variance, in which the second GP is learned by replicating every query for a fixed predetermined number of times. Importantly, all of these works on risk-averse BO have focused only on the sequential setting without support for batch evaluations. Replicating the selected inputs in BO multiple times has also been adopted by the recent works of [7, 13], which have shown that replication can lead to comparable or better theoretical and empirical performances of BO.

7 Conclusion

We have introduced the BTS-RED framework, which can trade-off between evaluating more unique conditions vs. replicating each condition more times and can perform risk-averse optimization. We derive theoretical guarantees for our methods to show that they are no-regret, and verify their empirical effectiveness in real-world precision agriculture and AutoML experiments. A potential limitation is that we use a heuristic (rather than principled) technique to handle unused budgets in an iteration (last paragraph of Sec. 3.1.1). Another interesting future work is to incorporate our technique of using an adaptive number of replications (depending on the noise variance) into other batch BO algorithms [18, 22] to further improve their performances. Moreover, it is also interesting to combine our method with the recent line of work on neural bandits [16, 17], which may expand the application of our method to more AI4Science problems.

References

F. Berkenkamp, A. P. Schoellig, and A. Krause. No-regret Bayesian optimization with unknown hyperparameters. Journal of Machine Learning Research , 2019.

M. Binois, J. Huang, R. B. Gramacy, and M. Ludkovski. Replication or exploration? sequential design for stochastic simulation experiments. Technometrics , 61(1):7-23, 2019.

I. Bogunovic, J. Scarlett, S. Jegelka, and V. Cevher. Adversarially robust optimization with Gaussian processes. In Proc. NeuIPS , 2018.

G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. OpenAI Gym. arXiv:1606.01540, 2016.

X. Cai and J. Scarlett. On lower bounds for standard and robust Gaussian process bandit optimization. In Proc. ICML , pages 1216-1226. PMLR, 2021.

S. Cakmak, R. Astudillo, P. Frazier, and E. Zhou. Bayesian optimization of risk measures. In Proc. NeurIPS , 2020.

D. Calandriello, L. Carratino, A. Lazaric, M. Valko, and L. Rosasco. Scaling Gaussian process optimization by evaluating a few unique candidates multiple times. In Proc. ICML , pages 2523-2541. PMLR, 2022.

S. R. Chowdhury and A. Gopalan. On kernelized multi-armed bandits. In Proc. ICML , pages 844-853, 2017.

S. R. Chowdhury and A. Gopalan. On batch Bayesian optimization. arXiv:1911.01032, 2019.

G. Cohen, S. Afshar, J. Tapson, and A. Van Schaik. EMNIST: Extending MNIST to handwritten letters. In Proc. IJCNN , pages 2921-2926. IEEE, 2017.

E. Contal, D. Buffoni, A. Robicquet, and N. Vayatis. Parallel Gaussian process optimization with upper confidence bound and pure exploration. In Proc. ECML/PKDD , pages 225-240, 2013.

A. I. Cowen-Rivers, W. Lyu, R. Tutunov, Z. Wang, A. Grosnit, R. R. Griffiths, A. M. Maraval, H. Jianye, J. Wang, J. Peters, et al. An empirical study of assumptions in Bayesian optimisation. arXiv:2012.03826, 2020.

Z. Dai, G. K. R. Lau, A. Verma, Y. Shu, B. K. H. Low, and P. Jaillet. Quantum Bayesian optimization. In Proc. NeurIPS , 2023.

Z. Dai, B. K. H. Low, and P. Jaillet. Federated Bayesian optimization via Thompson sampling. In Proc. NeurIPS , 2020.

Z. Dai, B. K. H. Low, and P. Jaillet. Differentially private federated Bayesian optimization with distributed exploration. In Proc. NeurIPS , volume 34, 2021.

Z. Dai, Y. Shu, B. K. H. Low, and P. Jaillet. Sample-then-optimize batch neural Thompson sampling. In Proc. NeurIPS , 2022.

Z. Dai, Y. Shu, A. Verma, F. X. Fan, B. K. H. Low, and P. Jaillet. Federated neural bandits. In Proc. ICLR , 2023.

E. A. Daxberger and B. K. H. Low. Distributed batch Gaussian process optimization. In Proc. ICML , pages 951-960, 2017.

T. Desautels, A. Krause, and J. W. Burdick. Parallelizing exploration-exploitation tradeoffs in Gaussian process bandit optimization. Journal of Machine Learning Research , 15:3873-3923, 2014.

D. Eriksson, M. Pearce, J. Gardner, R. D. Turner, and M. Poloczek. Scalable global optimization via local Bayesian optimization. In Proc. NeurIPS , 2019.

P. I. Frazier. A tutorial on Bayesian optimization. arXiv:1807.02811, 2018.

J. González, Z. Dai, P. Hennig, and N. Lawrence. Batch Bayesian optimization via local penalization. In Proc. AISTATS , pages 648-657. PMLR, 2016.

R.-R. Griffiths, A. A. Aldrick, M. Garcia-Ortegon, V. Lalchand, et al. Achieving robustness to aleatoric uncertainty with heteroscedastic Bayesian optimisation. Machine Learning: Science and Technology , 3(1):015004, 2021.

M. W. Hoffman, B. Shahriari, and N. de Freitas. Exploiting correlation and budget constraints in Bayesian multi-armed bandit optimization. arXiv:1303.6746, 2013.

F. Hutter, L. Kotthoff, and J. Vanschoren. Automated machine learning: methods, systems, challenges . Springer Nature, 2019.

S. Iwazaki, Y. Inatsu, and I. Takeuchi. Mean-variance analysis in Bayesian optimization under uncertainty. In Proc. AISTATS , pages 973-981. PMLR, 2021.

M. Jain, E. Bengio, A. Hernandez-Garcia, J. Rector-Brooks, B. F. Dossou, C. A. Ekbote, J. Fu, T. Zhang, M. Kilgour, D. Zhang, et al. Biological sequence design with gflownets. In proc. ICML , pages 9786-9801. PMLR, 2022.

K. Kandasamy, A. Krishnamurthy, J. Schneider, and B. Póczos. Parallelised Bayesian optimisation via thompson sampling. In Proc. AISTATS , pages 133-142. PMLR, 2018.

K. Kersting, C. Plagemann, P. Pfaff, and W. Burgard. Most likely heteroscedastic Gaussian process regression. In Proc. ICML , pages 393-400, 2007.

J. Kirschner and A. Krause. Information directed sampling and bandits with heteroscedastic noise. In Proc. COLT , pages 358-384. PMLR, 2018.

P. M. Kyveryga, T. A. Mueller, D. S. Mueller, D. Shannon, D. Clay, and N. Kitchen. On-farm replicated strip trials. Precis. Agric. Basics , pages 189-208, 2018.

B. Letham, R. Calandra, A. Rai, and E. Bakshy. Re-examining linear embeddings for highdimensional Bayesian optimization. In Proc. NeurIPS , volume 33, pages 1546-1558, 2020.

A. Makarova, I. Usmanova, I. Bogunovic, and A. Krause. Risk-averse heteroscedastic Bayesian optimization. Proc. NeurIPS , 34, 2021.

M. Mutn` y and A. Krause. Efficient high dimensional Bayesian optimization with additivity and quadrature Fourier features. In Proc. NeurIPS , pages 9005-9016. Curran, 2019.

E. Nava, M. Mutny, and A. Krause. Diversified sampling for batched bayesian optimization with determinantal point processes. In Proc. AISTATS , pages 7031-7054. PMLR, 2022.

Q. P. Nguyen, Z. Dai, B. K. H. Low, and P. Jaillet. Optimizing conditional value-at-risk of black-box functions. In Proc. NeurIPS , volume 34, 2021.

Q. P. Nguyen, Z. Dai, B. K. H. Low, and P. Jaillet. Value-at-risk optimization with Gaussian processes. In Proc. ICML , 2021.

V. Nguyen, S. Rana, S. K. Gupta, C. Li, and S. Venkatesh. Budgeted batch Bayesian optimization. In Proc. ICDM , pages 1107-1112. IEEE, 2016.

A. Rahimi, B. Recht, et al. Random features for large-scale kernel machines. In Proc. NeurIPS , volume 3, page 5. Citeseer, 2007.

C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning . MIT Press, 2006.

N. Srinivas, A. Krause, S. M. Kakade, and M. Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proc. ICML , pages 1015-1022, 2010.

S. S. Tay, C. S. Foo, U. Daisuke, R. Leong, and B. K. H. Low. Efficient distributionally robust Bayesian optimization with worst-case sensitivity. In Proc. ICML , pages 21180-21204. PMLR, 2022.

A. van Beek, U. F. Ghumman, J. Munshi, S. Tao, T. Chien, G. Balasubramanian, M. Plumlee, D. Apley, and W. Chen. Scalable adaptive batch sampling in simulation-based design with heteroscedastic noise. Journal of Mechanical Design , 143(3):031709, 2021.

A. Verma, Z. Dai, and B. K. H. Low. Bayesian optimization under stochastic delayed feedback. In Proc. ICML , pages 22145-22167. PMLR, 2022.

Z. Wang, C. Gehring, P. Kohli, and S. Jegelka. Batched large-scale Bayesian optimization in high-dimensional spaces. In Proc. AISTATS , pages 745-754. PMLR, 2018.

Z. Wang and S. Jegelka. Max-value entropy search for efficient Bayesian optimization. In Proc. ICML , pages 3627-3635. PMLR, 2017.

J. Wu and P. Frazier. The parallel knowledge gradient method for batch Bayesian optimization. In Proc. NeurIPS , volume 29, pages 3126-3134, 2016.

A Expressions of GP Posterior

Following the notations in the main text (Sec. 2), we index every queried input x [ b ] t and observed output y [ b ] t via an iteration index t and a batch index b . Define I t -1 the collection of indices of the queried inputs in the first t -1 iterations: I t -1 = { ( t ′ , b ) } t ′ ∈ [ t -1] ,b ∈ [ b t ′ ] . Note that according to our notations in the main text, the cardinality of I t -1 is |I t -1 | = τ t -1 = ∑ t -1 t ′ =1 b t ′ . Then, the GP posterior for the objective function f in iteration t can be represented as GP ( µ t -1 ( · ) , σ 2 t -1 ( · , · )) , where

in which k t -1 ( x ) ≜ [ k ( x , x [ b ] t ′ )] ⊤ ( t ′ ,b ) ∈I t -1 . y t -1 ≜ ( y [ b ] t ′ ) ⊤ ( t ′ ,b ) ∈I t -1 in which y [ b ] t ′ =

(1 /n [ b ] t ′ ) ∑ n [ b ] t ′ n =1 y [ b ] t ′ ,n represents the empirical mean at the input x [ b ] t ′ calculated using the n [ b ] t ′ replications. K t -1 ≜ ( k ( x [ b ] t ′ , x [ b ′ ] t ′′ )) ( t ′ ,b ) ∈I t -1 , ( t ′′ ,b ′ ) ∈I t -1 and λ > 0 is a regularization parameter and will need to be set to λ = 1 + 2 /T in order for our theoretical results to hold [8].

B Proof of Theorem 3.1

Denote by τ t -1 the total number of observations (input-output pairs) up to and including iteration t -1 : τ t -1 ≜ ∑ t -1 t ′ =1 b t ′ . This immediately implies that t -1 ≤ τ t -1 ≤ B ( t -1) . We use F t -1 to denote the history of all τ t -1 observations up to iteration t -1 . Denote by b t the batch size in iteration t . Note that conditioned on F t -1 , b t is a random variable, which is in contrast with standard batch BO in which the batch size is usually fixed. Here we use µ t -1 ( x ) and σ t -1 ( x ) to denote the GP posterior mean and standard deviation conditioned on all τ t -1 observations up to (and including) iteration t -1 . Moreover, denote by σ t -1 ,b ′ ( x ) the GP posterior standard deviation after additionally conditioning on the first b ′ = 0 , . . . , b t -1 selected inputs in iteration t . Note that σ t -1 , 0 ( x ) = σ t -1 ( x ) according to our definitions. Define β t ≜ B + R √ 2(Γ τ t -1 +1+log(2 /δ )) and c t ≜ β t (1 + √ 2 log( B |X| t 2 )) .

Lemma B.1. Let δ ∈ (0 , 1) . Define E f ( t ) as the event that | µ t -1 ( x ) -f ( x ) | ≤ β t σ t -1 ( x ) for all x ∈ X . We have that P [ E f ( t ) ] ≥ 1 -δ/ 2 for all t ≥ 1 .

Lemma B.1 is a consequence of Theorem 2 of the work of [8].

Lemma B.2. Define E f t ( t ) as the event: | f [ b ] t ( x ) -µ t -1 ( x ) | ≤ β t √ 2 log( B |X| t 2 ) σ t -1 ( x ) , ∀ x ∈ X , ∀ b ∈ [ b t ] . We have that P [ E f t ( t ) |F t -1 ] ≥ 1 -1 /t 2 for any possible filtration F t -1 .

Proof. According to Lemma B4 of [24], in iteration t , for a particular b and x , we have that

with probability of ≥ 1 -δ . Replacing δ by δ/ ( B |X| ) and taking a union bound over all x ∈ X and all b ∈ [ b t ] gives us:

which holds with probability of ≥ 1 -δ B |X| ×|X| b t ≥ 1 -δ , because b t ≤ B . Further replacing δ by 1 /t 2 completes the proof.

Next, we define the set of saturated points .

Definition B.3. Define the set of saturated points at iteration t as

in which ∆( x ) = f ( x ∗ ) -f ( x ) and x ∗ ∈ arg max x ∈X f ( x ) .

The next auxiliary lemma will be needed shortly to lower-bound the probability that an unsaturated point is selected.

LemmaB.4. For any filtration F t -1 , conditioned on the events E f ( t ) , we have that ∀ x ∈ X , b ∈ [ b t ] ,

in which p = 1 4 e √ π .

Proof. For any b ∈ [ b t ] , we have that

The second last inequality results from Lemma B.1, and the last inequality follows because f [ b ] t ( x ) follows a Gaussian distribution because f [ b ] t ∼ GP ( µ t -1 ( · ) , β 2 t σ 2 t -1 ( · )) . Lastly, since all f [ b ] t 's are sampled in the same way: f [ b ] t ∼ GP ( µ t -1 ( · ) , β 2 t σ 2 t -1 ( · )) , the proof above holds for all b ∈ [ b t ] .

The next lemma shows that the probability that an unsaturated input is selected can be lower-bounded.

Lemma B.5. For any filtration F t -1 , conditioned on the event E f ( t ) , we have that

Proof. For every b ∈ [ b t ] ,

which holds ∀ b ∈ [ b t ] . The validity of the inequality above can be seen by noting that x ∗ is always unsaturated , because ∆( x ∗ ) = f ( x ∗ ) -f ( x ∗ ) = 0 < c t σ t -1 ( x ) . As a result, if the event on the right hand side holds (i.e., if f [ b ] t ( x ∗ ) > f [ b ] t ( x ) , ∀ x ∈ S t ), then the event on the left hand side is guaranteed to hold because x [ b ] t is selected by x [ b ] t = arg max x ∈X f [ b ] t ( x ) which ensures that an unsaturated input will be selected.

Next, we assume that both events E f ( t ) and E f t ( t ) are true, which allows us to derive an upper bound on f [ b ] t ( x ) for all x ∈ S t and for all b ∈ [ b t ] :

where the first inequality follows from Lemma B.1 and Lemma B.2, and the second inequality results from Definition B.3. Therefore, (9) implies that for every b ∈ [ b t ] , if both both events E f ( t ) and E f t ( t ) hold, we have that

Next, conditioning only on the event E f ( t ) , for every b ∈ [ b t ] , we can show that

which holds for all b ∈ [ b t ] .

Next, we use the following Lemma to connect the GP posterior standard deviation given all observations in the first t -1 iterations (i.e., σ t -1 ( · ) ) with the conditional information gain from the selected input queries in the t th iteration (batch).

Lemma B.6. Define C 2 = 2 log(1+ λ -1 ) . Denote all τ t -1 observations from iterations (batches) 1 to t -1 as y 1: t -1 , and the b t observations in the t th batch as y t . Then we have that

Proof. Note that as has been described in the main text, the constant C is chosen such that:

Denote by y t, 1: b -1 the first b -1 observations within the t th batch, then for b > 1 ,

Also note that when b = 1 , σ t -1 ( x ) /σ t -1 ,b -1 ( x ) = 1 ≤ exp( C ) . Therefore, we have that

The second inequality makes use of the Cauchy-Schwarz inequality, and the last equality follows from the definition of information gain.

The next Lemma gives an upper bound on the expected batch regret in iteration t : min b ∈ [ b t ] r [ b ] t = min b ∈ [ b t ] ( f ( x ∗ ) -f ( x [ b ] t )) .

Lemma B.7. For any filtration F t -1 , conditioned on the event E f ( t ) , we have that

in which r [ b ] t = f ( x ∗ ) -f ( x [ b ] t )

Proof. To begin with, we define x t as the unsaturated input at iteration t (after the first t -1 iterations) with the smallest (posterior) standard deviation:

Following this definition, for any F t -1 such that E f ( t ) is true, ∀ b ∈ [ b t ] , we have that

Now we condition on both events E f ( t ) and E f t ( t ) , and analyze the instantaneous regret as:

in which the second inequality results from Lemma B.1 and Lemma B.2, the third inequality follows since x t is unsaturated, and the last inequality follows from the policy in which x [ b ] t is selected, i.e., x [ b ] t = arg max x ∈X f [ b ] t ( x ) .

Now we separately consider the two cases where the event E f t ( t ) is true and false:

where the second inequality follows from equation (16), the fourth inequality results from Lemma B.6, and the last inequality follows since 2 p -1 /t 2 ≤ 10 p .

Definition B.8. Define Y 0 = 0 , and for all t = 1 , . . . , T ,

Lemma B.9. Conditioned on Lemma B.7 (i.e., with probability of ≥ 1 -δ/ 2 ), ( Y t : t = 0 , . . . , T ) is a super-martingale with respect to the filtration F t .

Proof.

When the event E f ( t ) holds, then r t = min b ∈ [ b t ] r [ b ] t and the inequality follows from Lemma B.7; when E f ( t ) does not hold, r t = 0 and hence the inequality trivially holds.

Lemma B.10. Define C 0 ≜ 1 B / ⌈ σ 2 max R 2 ⌉-1 . Given δ ∈ (0 , 1) , then with probability of at least 1 -δ ,

Proof. To begin with, let's derive a lower bound on b t . Firstly, note that n t ≤ ⌈ σ 2 max R 2 ⌉ . This implies that b t ≥ B / ⌈ σ 2 max R 2 ⌉ -1 . Next, we need to derive an upper bound on | Y t -Y t -1 | which will be used when we apply the Azuma-Hoeffding's inequality:

where the second inequality follows because

Next, we will need the following result to connect the sum of condition information gains to the maximum information gain:

in which the first inequality results from the Cauchy-Schwarz inequality, the second inequality follows from the chain rule of conditional information gain, and the last inequality makes use of the

fact that τ T ≤ T B . Subsequently, we are ready to upper-bound the batch cumulative regret:

The first inequality results from the Azuma-Hoeffding's inequality, the second inequality makes use of the lower bound on b t : b t ≥ B / ⌈ σ 2 max R 2 ⌉ -1 , and the last inequality follows from equation (21). Equation (22) holds with probability of ≥ 1 -δ/ 2 according to the Azuma-Hoeffding's inequality. Then, we also have that r t = r t , ∀ t ≥ 1 with probability of ≥ 1 -δ/ 2 according to Lemma B.1. This completes the proof.

Note that c T = ˜ O ( R √ Γ T B ) , which allows us to simplify the regret upper bound into an asymptotic expression:

B.1 Simplified Regret Upper Bound for Constant Noise Variance

In the special case where the noise variance is fixed throughout the entire domain, i.e., when σ 2 ( x ) = σ 2 const , ∀ x ∈ X , we have that n t = ⌈ σ 2 const /R 2 ⌉ = n const and b t = ⌊ B /n const ⌋ = b 0 , ∀ t ∈ [ T ] . In this case, instead of making use of the lower bound on b t (i.e., b t ≥ B / ⌈ σ 2 max R 2 ⌉ -1 ) as done in the proof of Lemma B.10, we can simply replace b t with b 0 in the proof. As a result, the term of 1 √ B / ⌈ σ 2 max R 2 ⌉-1

in the regret upper bound proved in Lemma B.10 can be simply replaced by 1 √ b 0 , and hence the regret upper bound can be further simplified as:

As another special case where σ 2 ( x ) = σ 2 const = R 2 , every input x will be evaluated only once (i.e., n const = 1 ) and b 0 = B . In this case, our algorithm reduces to the standard batch TS with a batch size of B , and the regret upper bound becomes

C Choice of R 2 by Minimizing the Regret Upper Bound in Theorem 3.1

The regret bound in Theorem 3.1 depends on the parameter R through the term g ≜ √ R 2 B / ( σ 2 max /R 2 +1) -1 , in which we have replaced the term ⌈ σ 2 max /R 2 ⌉ by σ 2 max /R 2 +1 such that the

resulting regret upper bound is still valid and the subsequent derivations become simplified. Taking the derivative of g 2 w.r.t. R 2 gives us

Setting the above derivative to 0 , we have that the value of R 2 that minimizes g 2 and g (hence minimizing the regret upper bound) is obtained at

D Upper Bound on Sequential Cumulative Regret

E Confidence Bound for the Noise Variance Function

Here, as discussed in Sec. 3.2 in the main text, we leverage the concentration of the Chi-squared distribution to show that we can interpret the unbiased estimate of the noise variance in equation (1) as a noisy observation corrupted by a sub-Gaussian noise. For every queried input, we use the unbiased empirical variance (1) as the observation to update the GP posterior for the noise variance. First of all, when an input x [ b ] t is queried, denote the unbiased empirical variance as ˜ σ 2 ( x [ b ] t ) = σ 2 ( x [ b ] t ) + ϵ ′ , and we will show next that ϵ ′ is (with high probability) a sub-Gaussian noise. Denote as n t the number replications that we use to query x [ b ] t . Since the unbiased empirical variance of a Gaussian random variable follows a Chi-squared distribution, we have that

( χ 2 n t -1 , 1 -α/ 2 ) denotes the ( α/ 2) th -quantile ( (1 -α/ 2) th -quantile) of the Chi-square distribution with n t -1 degrees of freedom. This allows us to show that

which holds with probability ≥ 1 -α . n min is the minimum number of repetitions we impose on every queried x t . Note that the discussion above on the unbiasedness and boundedness of the noise ϵ ′ still holds after we negate the noise variance (i.e., g ( · ) = -σ 2 ( · ) ) to analyze ˜ y [ b ] t = g ( x [ b ] t ) + ϵ ′ . Then the discussion in the main text (Sec. 3.2) follows.

F The BTS-RED-Unknown Algorithm

Our BTS-RED-Unknown algorithm is shown in Algorithm 3.

G Proof of Theorem 4.1

Our proof here follows a similar structure to the proof in Appendix B, but non-trivial changes need to be made to account for the change of objective function here. Here, for a given ω , we attempt to upper-bound the mean-variance cumulative regret R MV T = ∑ T t =1 min b ∈ [ b t ] [ h ω ( x ∗ ω ) -h ω ( x [ b ] t )] , where h ω ( x ) = ωf ( x ) + (1 -ω ) g ( x ) = ωf ( x ) -(1 -ω ) σ 2 ( x ) and x ∗ ω ∈ arg max x ∈X [ ωf ( x ) + (1 -ω ) g ( x ) ] . Throughout the analysis here, we assumed a pre-defined ω and hence omit any dependence on ω for simplicity. That is, we use x ∗ in place of x ∗ ω and use h ( · ) to represent h ω ( · ) .

Algorithm 3 BTS-RED-Unknown.

1: for t = 1 , 2 , . . . , T do

2: b = 0 , n (0) t = 0

4: b ← b +1

3: while ∑ b b ′ =0 n ( b ′ ) t < B do

5: Sample f ( b ) t from GP ( µ t -1 ( · ) , β 2 t σ 2 t -1 ( · , · ))

6: Choose x ( b ) t = arg max x ∈X f ( b ) t ( x )

7: Choose n ( b ) t = ⌈ ( -µ ′ t -1 ( x ( b ) t ) + β ′ t σ ′ t -1 ( x ( b ) t )) /R 2 ⌉

8: b t = b -1

9: for b ∈ [ b t ] , query x ( b ) t with n ( b ) t parallel processes

10: for b ∈ [ b t ] , observe { y ( b ) t,n } n ∈ [ n ( b ) t ] , then calculate y ( b ) t = (1 /n ( b ) t ) ∑ n ( b ) t n =1 y ( b ) t,n and ˜ y ( b ) t = -1 / ( n ( b ) t -1) ∑ n ( b ) t n =1 ( y ( b ) t,n -y ( b ) t ) 2

11: Use { ( x ( b ) t , y ( b ) t ) } b ∈ [ b t ] to update posterior of GP

12: Use { ( x ( b ) t , ˜ y ( b ) t ) } b ∈ [ b t ] to update posterior of GP ′

Here, same as the main text, we use g : X → R -to denote the function -σ 2 : X → R -. We use µ t -1 and σ t -1 to denote the GP posterior mean and standard deviation of the GP for f conditioned on all τ t -1 observations up to (and including) iteration t -1 , and use µ ′ t -1 and σ ′ t -1 to denote the GP posterior mean and standard deviation of the GP for the negative noise variance -σ 2 . Denote h [ b ] t ( x ) = ωf [ b ] t ( x ) + (1 -ω ) g [ b ] t ( x ) , such that x [ b ] t ∈ arg max x ∈X h [ b ] t . Denote by F ′ t -1 the history of observed pairs of input and empirical noise variance up to iteration t -1 .

Define β t ≜ B + R √ 2(Γ τ t -1 +1+log(3 /δ )) and c t ≜ β t (1 + √ 2 log(2 B |X| t 2 )) . Also define β ′ t ≜ B ′ + R ′ √ 2(Γ ′ τ t -1 +1+log(3 /δ )) and c ′ t ≜ β ′ t (1 + √ 2 log(2 B |X| t 2 )) .

Lemma G.1. Let δ ∈ (0 , 1) . Define E g ( t ) as the event that | µ ′ t -1 ( x ) -g ( x ) | ≤ β ′ t σ ′ t -1 ( x ) for all x ∈ X . We have that P [ E g ( t ) ] ≥ 1 -δ/ 3 for all t ≥ 1 .

The validity of Lemma G.1 follows from the discussion in Sec. 3.2 (i.e., equation (G.1)), after replacing the error probability of δ/ 2 by δ/ 3 . We also need Lemma B.1 to hold, and since we have replaced the error probability of δ/ 2 in β t from Lemma B.1 by δ/ 3 in our definition of β t above, we have that the event E f ( t ) in Lemma B.1 holds with probability of ≥ 1 -δ/ 3 here.

Lemma G.2. Define E g t ( t ) as the event: | g [ b ] t ( x ) -µ ′ t -1 ( x ) | ≤ β ′ t √ 2 log(2 B |X| t 2 ) σ ′ t -1 ( x ) , ∀ b ∈ [ b t ] . We have that P [ E g t ( t ) |F t -1 ] ≥ 1 -1 / (2 t 2 ) for any possible filtration F ′ t -1 .

Similarly, we will also need Lemma B.2 to hold, but replace the error probability of 1 /t 2 by 1 / (2 t 2 ) . Of note, we have also correspondingly changed the value of c t from Appendix B by replacing t 2 by 2 t 2 in our definition of c t above.

The definition of saturated points also needs to be modified:

Definition G.3. Define the set of saturated points at iteration t as

in which ∆( x ) = h ( x ∗ ) -h ( x ) and x ∗ ∈ arg max x ∈X h ( x ) .

The following lemma is a counterpart to Lemma B.4 in Appendix B, and the proof here makes use of the same techniques.

Lemma G.4. For any F ′ t -1 , conditioned on the events E g ( t ) , we have that ∀ x ∈ X , b ∈ [ b t ] ,

Proof. For any b ∈ [ b t ] , we have that

The next Lemma is the counterpart to Lemma B.5 in Appendix B, but additional challenges need to be carefully handled here.

Lemma G.5. For any F t -1 and F ′ t -1 , conditioned on the events E f ( t ) and E g ( t ) , we have that

Proof. For every b ∈ [ b t ] ,

which holds ∀ b ∈ [ b t ] . This inequality follows from noting that x ∗ is always unsaturated according to Definition G.3, and that x [ b ] t = arg max x ∈X h [ b ] t ( x ) = arg max x ∈X ( ωf [ b ] t ( x ) + (1 -ω ) g [ b ] t ( x )) .

Next, we assume that the events E f ( t ) , E f t ( t ) , E g ( t ) and E g t ( t ) all hold, which allows us to derive an upper bound on h [ b ] t ( x ) for all x ∈ S ′ t and for all b ∈ [ b t ] :

where the first inequality follows from Lemmas B.1, B.2, G.1 and G.2, and the second inequality makes use of the definition of h ( x ) as well as Definition G.3.

Therefore, (32) implies that for every b ∈ [ b t ] , if the events E f ( t ) , E f t ( t ) , E g ( t ) and E g t ( t ) all hold, then we have that

The second inequality results since the event in the third line implies the event in the line above, and the last inequality follows from Lemmas B.4 and G.4.

Next, for every b ∈ [ b t ] , we can show that

which holds for all b ∈ [ b t ] . The second inequality follows from considering the following two events separately: E f t ( t ) ∩ E g t ( t ) and E f t ( t ) ∩ E g t ( t ) , and the last inequality follows since P ( E f t ( t ) ∪ E g t ( t ) |F t -1 , F ′ t -1 ) ≤ P ( E f t ( t ) |F t -1 , F ′ t -1 ) + P ( E g t ( t ) |F t -1 , F ′ t -1 ) = 1 / (2 t 2 ) + 1 / (2 t 2 ) = 1 /t 2 .

Next, we will need the following Lemma which is a counterpart to Lemma B.6.

Lemma G.6. Define C 2 = 2 log(1+ λ -1 ) . Denote all τ t -1 observed empirical means from iterations (batches) 1 to t -1 as y 1: t -1 , and the b t observed empirical means in the t th batch as y t . Also denote all τ t -1 observed noise variances from iterations (batches) 1 to t -1 as y ′ 1: t -1 , and the b t observed noise variances in the t th batch as y ′ t . Choose C as an absolute constant such that max A ⊂X , | A |≤ B I ( f ; y A | y 1: t -1 ) ≤ C, ∀ t ≥ 1 and max A ⊂X , | A |≤ B I ( g ; y ′ A | y ′ 1: t -1 ) ≤ C, ∀ t ≥ 1 . Then we have that

Proof. Note that we have assumed that both f and g are associated with the SE kernels k and k ′ , respectively. Denote the length scales for k and k ′ by θ and θ ′ respectively. Note that the maximum information gain is decreasing in the length scale, i.e., a smaller length scale leads to a larger maximum information gain [1]. Therefore, we can run the initialization stage via uncertainty sampling to observe y init using the kernel with a smaller length scale.

For example, if θ < θ ′ , we use the kernel k to run the uncertainty sampling algorithm for T init iterations to collect the initial set of inputs D init, such that we can guarantee that

where the first inequality follows from the submodularity of conditional information gain, and the second inequality is a consequence of Lemma 4 of [19]. Note that given the same set of initial inputs D init, the maximum conditional information gains max A ⊂X , | A |≤ B I ( f ; y A | y init ) and max A ⊂X , | A |≤ B I ( g ; y ′ A | y ′ init ) differ by only the lengthscales of the kernels k and k ′ , denoted as θ and θ ′ , respectively. Therefore, since we have assumed that θ < θ ′ in the discussion here, we have that

This further tells us that

Subsequently, the proof is completed by applying the proof techniques of Lemma B.6 to σ t -1 and σ ′ t -1 separately.

and

Lemma G.7. Define ˜ B = ωB +(1 -ω ) B ′ . For any filtrations F t -1 and F ′ t -1 , conditioned on the events E f ( t ) and E g ( t ) , we have that

in which r [ b ] t = h ( x ∗ ) -h ( x [ b ] t ) .

Proof. To begin with, we define x t as the unsaturated input at iteration t with the smallest weighted posterior standard deviation:

Following this definition, if both E f ( t ) and E g ( t ) hold ∀ b ∈ [ b t ] , then we have that

Now we condition on all events E f ( t ) , E f t ( t ) , E g ( t ) and E g t ( t ) , and analyze the instantaneous regret as:

in which the second inequality follows from Lemmas B.1, B.2, G.1 and G.2, the third inequality results from Definition G.3 and the fact that x t is unsaturated, and the last inequality follows from the way in which x [ b ] t is selected: x [ b ] t = arg max x ∈X h [ b ] t ( x ) .

The second inequality follows from equation (39), the third inequality follows since 1 / ( p 2 -1 /t 2 ) ≤ 14 /p 2 , and the last inequality makes use of Lemma G.6.

Following the proof of Lemma B.9, we can easily show that ( Y t : t = 0 , . . . , T ) is a super-martingale. Now we are finally ready to prove an upper bound on the batch cumulative regret of our Mean-VarBTS-RED:

in which Γ T B is the maximum information gain about f obtained from any set of T B observations, and Γ ′ T B is the maximum information gain about g obtained from any set of T B observations.

Proof. To begin with, let's derive a lower bound on b t . Firstly, note that n t ≤ ⌈ σ 2 max R 2 ⌉ . This implies that b t ≥ B / ⌈ σ 2 max R 2 ⌉ -1 .

Applying the Azuma-Hoeffding's inequality using an error probability of δ/ 3 leads to

The second inequality makes use of equation (43) and the lower bound on b t : b t ≥ B / ⌈ σ 2 max R 2 ⌉ -1 . Now, note that r t = r t , ∀ t ≥ 1 with probability of ≥ 1 -δ/ 3 -δ/ 3 because both events E f ( t ) and E g ( t ) hold with probability of ≥ 1 -δ/ 3 respectively. As a result, taking into account the error probability of δ/ 3 from the Azuma-Hoeffding's inequality, the regret upper bound holds with probability of ≥ 1 -δ/ 3 -δ/ 3 -δ/ 3 = 1 -δ .

0.0

0.2

0.4

0.6

0.8

1.0

input

0.00

0.05

0.10

0.15

0.20

0.25

upper bound

Finally, we can simplify the regret upper bound from Lemma G.9 into asymptotic notation:

H More Experimental Details

Since the theoretical value of β t and β ′ t is usually too conservative [41, 3], we follow the common practice and set them to a constant: β t = β ′ t = 1 . In every experiment, we use the same set of initial inputs selected via random search for all methods to ensure a fair comparison. For all methods based on TS (including all our methods), i.e., all methods which require sampling functions from the GP posterior (e.g., line 5 of Algo. 1, line 5 of Algo. 3 and line 5 of Algo. 2), we follow the common practice and sample the functions from the GP posterior through random Fourier features approximation [39, 46, 34]. Again following the common practice in BO, we optimize the GP hyperparameters by maximizing the marginal likelihood after every 10 iterations. Our experiments are run on a computer server with 128 CPUs, with the AMD EPYC 7543 32-Core Processor. The server has 8 NVIDIA GeForce RTX 3080 GPUs.

H.1 Synthetic Experiments

In the synthetic experiments, for both mean and mean-variance optimization, we firstly use a sampled function from a GP with the SE kernel (lengthscale = 0 . 04 ) as the objective function f (normalized into the range [0 , 1] ), and then sample another function from a GP with the SE kernel (lengthscale = 0 . 15 ) as the noise variance function σ 2 (normalized into the range [0 . 0001 , 0 . 2] ).

Fig. 4 shows an example of the upper bound U σ 2 t ( x ) = -µ ′ t -1 ( x ) + β ′ t σ ′ t -1 ( x ) constructed by our BTS-RED-Unknown (Sec. 3.2), which is an effective approximation of the groundtruth σ 2 ( · ) .

Here we also use the synthetic experiments to explore the impact of the uncertainty sampling (US) initialization, which is required by our theoretical results (Sec. 3.1.2), affects the empirical performance of our algorithms. The results in Fig. 8 show that using US and random search as the initialization method lead to similar performances. This provides an empirical justification for our choice of using the simpler random search as the initialization method in our main experiments.

H.2 Real-world Experiments on Plant Growth

As we have mentioned in the main text (Sec. 5.2), we perform real-world experiments using the input conditions from a regular a 2-D grid within the 2-D domain. Specifically, the 2-D grid is constructed using the pH values of { 2 . 5 , 3 . 0 , 3 . 5 , 4 . 0 , 4 . 5 , 5 . 5 , 6 . 5 } and NH3 concentrations of { 0 , 1 , 5 , 10 , 15 , 20 , 25 , 30 } ( × 1000 uM). In other words, the size of the grid is 7 × 8 = 56 . Every tested input condition is replicated 6 times. For every tested input condition, the leaf area and tipburn area after harvest are observed, both measured in the unit of mm 2 . The resulting observations are

0

20

40

60

Iterations

0.00

0.05

0.10

0.15

Simple Regret

Batch TS (

nt

=1)

Batch TS (

nt

=5)

Batch TS (

nt

=10)

Batch TS (

nt

=20)

BTS-RED-Unknown (

κ

=0.3)

BTS-RED-Unknown (

κ

=0.2)

GP-TS

GP-UCB

GP-UCB (heteroscedastic GP)

0.0

0.2

0.4

0.6

0.8

1.0

input

-0.5

0.0

0.5

1.0

1.5

0.0

0.2

0.4

0.6

0.8

1.0

input

0.0

0.2

0.4

0.6

0.8

1.0

Mean Objective

Mean-Var Objective

(a)

(b)

0

10

20

30

40

50

60

70

(a)

Iterations

0.00

0.05

0.10

0.15

Simple Regret

Different

κ

BTS-RED-Known (

κ

=0.2)

BTS-RED-Known (

κ

=0.3)

BTS-RED-Known (

κ

=0.4)

BTS-RED-Known (

κ

=0.5)

BTS-RED-Known (

κ

=0.6)

BTS-RED-Known (

κ

=0.8)

0

10

20

30

40

50

60

70

Iterations

0.00

0.02

0.04

0.06

0.08

0.10

0.12

Simple Regret

Batch TS (

nt

=1)

Batch TS (

nt

=5)

Batch TS (

nt

=10)

Batch TS (

nt

=20)

BTS-RED (

κ

=0.3)

BTS-RED (

κ

=0.2)

(a)

(b)

0

20

40

60

Iterations

0.00

0.05

0.10

0.15

Simple Regret

Uncertainty Sampling

BTS-RED-Unknown (

κ

=0.3) with US

BTS-RED-Unknown (

κ

=0.3)

BTS-RED-Unknown (

κ

=0.2) with US

BTS-RED-Unknown (

κ

=0.2)

2.5

3.3

4.1

4.9

5.7

6.5

pH

0

6

12

18

24

30

NH3 (×1000)

8

6

5

10

10

7

5

10

8

16

5

26

6

5

7

5

11

6

7

5

5

5

5

19

5

21

11

5

20

12

20

5

5

7

5

11

5

16

16

22

6

14

5

8

5

5

6

5

6

11

5

6

5

6

5

8

8

5

5

5

5

15

8

5

6

5

5

5

5

7

5

6

5

21

21

5

5

8

9

7

10

5

5

8

5

5

5

5

7

7

5

7

18

5

11

23

6

8

5

7

11

7

20

5

7

5

5

5

7

9

5

5

6

5

5

7

5

16

5

10

5

6

8

5

12

7

5

5

5

7

6

5

7

5

5

5

5

5

5

5

7

7

5

5

21

7

5

5

6

5

5

6

5

5

5

6

5

10

5

7

8

Mean-Var Objective

100

50

0

50

2.5

3.3

4.1

4.9

5.7

6.5

pH

0

6

12

18

24

30

NH3 (×1000)

8

6

5

10

10

7

5

10

8

16

5

26

6

5

7

5

11

6

7

5

5

5

5

19

5

21

11

5

20

12

20

5

5

7

5

11

5

16

16

22

6

14

5

8

5

5

6

5

6

11

5

6

5

6

5

8

8

5

5

5

5

15

8

5

6

5

5

5

5

7

5

6

5

21

21

5

5

8

9

7

10

5

5

8

5

5

5

5

7

7

5

7

18

5

11

23

6

8

5

7

11

7

20

5

7

5

5

5

7

9

5

5

6

5

5

7

5

16

5

10

5

6

8

5

12

7

5

5

5

7

6

5

7

5

5

5

5

5

5

5

7

7

5

5

21

7

5

5

6

5

5

6

5

5

5

6

5

10

5

7

8

3

4

5

6

7

8

9

(a)

(b)

0

20

40

60

80

100

Iterations

120

125

130

135

140

Leaf Area

Batch TS (

nt

=5)

Batch TS (

nt

=10)

Batch TS (

nt

=20)

BTS-RED-Unknown (

κ

=0.3)

BTS-RED-Unknown (

κ

=0.2)

GP-TS

GP-UCB (heteroscedastic GP)

GP-UCB

0

20

40

60

80

100

Iterations

90

95

100

105

110

Leaf Area - Tipburn Area

Batch TS (

nt

=5)

Batch TS (

nt

=10)

Batch TS (

nt

=20)

BTS-RED-Unknown (

κ

=0.3)

BTS-RED-Unknown (

κ

=0.2)

GP-TS

GP-UCB (heteroscedastic GP)

GP-UCB

(a)

(b)

used to learn a heteroscedastic GP model from GPglow ( https://gpflow.readthedocs.io/en/ develop/notebooks/advanced/heteroskedastic.html ), which is then used to produce the groundtruth mean and variance function used in our experiments.

H.3 Real-world Experiments on AutoML

In this experiment, we aim to tune two hyperparameters of SVM: the penalty parameter (denoted as C ) and RBF kernel parameter (referred to as gamma), both within the range of [0 . 0001 , 2] . To obtain the groundtrugh mean and variance functions, we firstly construct a uniform 2-D grid of size 80 × 80 using the 2-D domain, and then evaluate every input hyperparameter configuration on the grid using 100 different classification tasks (i.e., using the images of hand-written characters from 100 different individuals in the EMNIST dataset). The EMNIST dataset is under the CC0 license. For every tested input hyperparameter configuration on the grid, we record the empirical mean and variance as the groundtruth mean and variance at the corresponding input location. As a result, this completes our construction of the groundtruth mean and variance functions with the input domain being discrete with a size of 80 × 80 = 6400 .

Figs. 11a and b plot the constructed groundtruth mean and noise variance functions, including the locations of some selected inputs (the selected inputs after every 4 iterations are included) as well as their corresponding number of replications n t 's. Similarly, Figs. 11c and d show the queried inputs and the n t 's of Mean-Var-BTS-RED ( ω = 0 . 2 ), shown on the heat maps of the mean-variance objective (c) and noise variance functions (d). The figures show that similar to Fig. 3 for the precision agriculture experiment, the majority of our input queries fall into regions with large (either mean or

Mean

0.00

0.48

0.99

1.49

2.00

C

0.00

0.48

0.99

1.49

2.00

gamma

7

7

13

13

11

12

12

13

8

8

8

9

6

9

9

7

9

6

5

5

7

5

4

7

8

9

7

9

4

7

4

10

4

7

6

5

9

12

5

11

10

10

4

8

4

4

7

12

5

8

6

11

6

7

10

7

12

4

10

11

7

4

4

10

8

11

6

12

8

9

11

5

9

8

9

8

11

10

7

7

9

6

11

8

5

6

7

5

8

7

10

11

4

7

7

8

7

9

8

10

10

7

11

3

6

9

5

10

8

7

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

0.00

0.48

0.99

1.49

2.00

C

0.00

0.48

0.99

1.49

2.00

gamma

7

7

13

13

11

12

12

13

8

8

8

9

6

9

9

7

9

6

5

5

7

5

4

7

8

9

7

9

4

7

4

10

4

7

6

5

9

12

5

11

10

10

4

8

4

4

7

12

5

8

6

11

6

7

10

7

12

4

10

11

7

4

4

10

8

11

6

12

8

9

11

5

9

8

9

8

11

10

7

7

9

6

11

8

5

6

7

5

8

7

10

11

4

7

7

8

7

9

8

10

10

7

11

3

6

9

5

10

8

7

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

(a)

(b)

0.00

0.48

0.99

1.49

2.00

C

0.00

0.48

0.99

1.49

2.00

gamma

9

22

11

6

14

5

6

6

5

8

6

8

5

4

5

4

5 5

4

5 5

14

7

7

10

3

6

4

3

21

12

12

15

3

3

4

6

10

4

4

7

3

3

4

7

9

3

6

10

5

2

13

6

9

5

15

3

13

6

9

4

5

6

6

3

17

16

5

7

4

11

13

3

4

16

13

6

11

5

11

7

3

9

5

8

12

6

10

12

10

6

5

10

3

3

9

9

9

12

3

6

13

5

13

11

4

11

3

5

8

3

6

4

8

4

7

8 7

5

6

6

7

7

7

Mean-Var Objective

0.02

0.04

0.06

0.08

0.10

0.12

0.14

0.16

0.00

0.48

0.99

1.49

2.00

C

0.00

0.48

0.99

1.49

2.00

gamma

9

22

11

6

14

5

6

6

5

8

6

8

5

4

5

4

5 5

4

5 5

14

7

7

10

3

6

4

3

21

12

12

15

3

3

4

6

10

4

4

7

3

3

4

7

9

3

6

10

5

2

13

6

9

5

15

3

13

6

9

4

5

6

6

3

17

16

5

7

4

11

13

3

4

16

13

6

11

5

11

7

3

9

5

8

12

6

10

12

10

6

5

10

3

3

9

9

9

12

3

6

13

5

13

11

4

11

3

5

8

3

6

4

8

4

7

8 7

5

6

6

7

7

7

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

(c)

(d)



0

20

40

60

Iterations

0.00

0.01

0.02

0.03

0.04

Simple Regret

Larger budget

=100

Batch TS (fixed

nt

=5)

Batch TS (fixed

nt

=10)

Batch TS (fixed

nt

=20)

Batch TS (fixed

nt

=40)

BTS-RED-Unknown (

κ

=0.3)

BTS-RED-Unknown (

κ

=0.2)

0

20

40

60

Iterations

0.00

0.01

0.02

0.03

0.04

0.05

0.06

Simple Regret

Smaller budget

=30

Batch TS (fixed

nt

=5)

Batch TS (fixed

nt

=10)

Batch TS (fixed

nt

=15)

BTS-RED-Unknown (

κ

=0.3)

BTS-RED-Unknown (

κ

=0.2)



mean-variance) objective function values (Figs. 11a and c) and that the selected number of replications n t is generally larger at those input locations with larger noise variance (Figs. 11b and d).

H.4 Additional Experiments with Higher-Dimensional Inputs

The Lunar-Lander experiment requires tuning d = 12 parameters of a heuristic controller used to control the Lunar-Lander environment from OpenAI Gym [4]. The controller can be found at https: //github.com/openai/gym/blob/8a96440084a6b9be66b2216b984a1c170e4a061c/gym/ envs/box2d/lunar_lander.py#L447 . The robot pushing task was introduced by [45], and we defer a more detailed introduction to the experimental settings to the work of [45]. Both experiments are widely used benchmarks for high-dimensional BO experiments [16, 20]. In both experiments, for

0

20

40

60

80

100

Iterations

-0.2

0.0

0.2

0.4

0.6

Cumulative rewards

12-dimensional

Batch TS (

nt

=5)

Batch TS (

nt

=10)

Batch TS (

nt

=20)

BTS-RED-Unknown (

κ

=0.3)

BTS-RED-Unknown (

κ

=0.2)

0

20

40

60

80

100

Iterations

2

3

4

5

6

Cumulative rewards

14-dimensional

Batch TS (

nt

=5)

Batch TS (

nt

=10)

Batch TS (

nt

=20)

BTS-RED-Unknown (

κ

=0.3)

BTS-RED-Unknown (

κ

=0.2)

(a) Lunar landing.

(b) Robot pushing.

every evaluated controller parameters (i.e., every evaluated input x ), the observation (i.e., cumulative rewards) is noisy due to random environmental factors. In addition, the noise may be heteroscedastic. For example, an effective set of parameters which can reliably and consistently control the robot is likely to induce small noise variance, whereas some ineffective sets of parameters may cause radically varying behaviors and hence large noise variances. Therefore, these experiments are also suitable for the application of our algorithms. Since the domain is continuous in these two experiments, here we maximize the acquisition function in every iteration via a combination of random search and L-BFGS-B: in every iteration when we need to maximize the acquisition function, we firstly randomly sample 10 , 000 inputs/points in the entire domain and find the point with the largest acquisition function value, and then further refine the search via L-BFGS-B with 100 random re-starts.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a batch Bayesian optimization method for the setting where experiments can be very noisy, and therefore it is common practice to repeat many experiments. The framework allows for both the selection of the experiment design and how often each experiment is replicated. The authors introduce three method: BTS-RED-Known (for the setting where the noise is known), BTS-RED-Unknown (for the setting where the noise is unknown), and Mean-Var-BTS-RED, for the setting where we want to trade-off finding the optimum while obtaining less noisy observations (called risk-averse optimization). \n\nIn every case the experiment designs are chosen using a variation of Thompson Sampling (with scaled variance). The number of replications is then chosen based on the noise level at the experiment design, an upper-bound is used when the noise is unknown. For risk-averse optimization, a linearized sum of the objective and the noise level is used. For every algorithm theoretical bounds are found on the regret, which are proven to be sub-linear. The number of repeated experiments depends on the hyper-parameters $R^2$, but a theoretical justification for its value is given. \n\nEmpirically, the method is shown to outperform other Bayesian Optimization algorithms in synthetic examples, and two real world examples: one in Precision Agriculture, and one in AutoML.\n\nSTRENGTHS:\nOriginality: To the best of my knowledge, the proposed algorithm is novel. While it combines simple ideas, it does so carefully and effectively. In particular, the theoretical analysis of the algorithms is very strong, providing a choice for the hyper-parameter $R^2$, and proving regret bounds for the algorithm.\n\nQuality: The algorithm is built up from proved and tested methods in the literature, and a strong theoretical justification is given for using them. Empirical evidence of the algorithm performance is given, but in a limited capacity. The creating of a data-set (for the agriculture example) based on real-life experiments makes the experiment particularly strong. \n\nClarity: The method is well explained, and the theoretical implications are clear. There is a good choice of figures, even if they are difficult to read. \n\nSignificance: The paper is well motivated, experiment replication due to noise is common in many areas of science and it is not usually taken into account by classical Bayesian optimization algorithms. Making BO more practical to the wider scientific community is very important, and papers like this one take a big step towards it.\n\nWEAKNESSES:\nWhile the motivation, and justification for the algorithms is clear, I find the empirical evidence to be lacking. My main concerns are:\n\n- The main framework seems to fit a _homoskedastic_ GP to the mean of the replicated data (which will, by design, have very small noise so it should not be a problem). Then, if needed, a second homoskedastic GP is used to model the variance. However, I would argue the most naive and natural solution is to simply use a heteroskedastic GP as the surrogate model in the first place. This is not a problem in itself, however, I believe for a fair comparison against Thompson Sampling a homoskedastic GP should be used. Otherwise the model misspecification could be problematic and lead to poor performance (maybe Figure 1d could be explained by this?). It is unclear from the paper if this is the case. In simpler terms, *it is unclear wether the empirical advantage of BTS-RED comes from a better algorithm, or a better model*. If a heterskedastic GP is used, then is should be clarified, otherwise the paper would benefit greatly from including it.\n\n- The extent of the experiments seems limited. Only a single 1d example is shown, and then both real world examples are 2d. It would be good to include more examples. Similarly, the budget is fixed to $\\mathbb{B} = 50$ for all experiments, it would be good to see how the algorithm performs for smaller (and perhaps larger) budgets. \n\nOther minor issues:\n\n- The novelty of Mean-Var-BTS-RED seems overstated, is it simply solving a multi-objective problem through linearization? \n\n- The heuristic for handling unused budget seems a little problematic. Given you get some observations (albeit noisy), if they clearly pointed to a sub-optimal design it seems wasteful to still evaluate the objective further at the specific location.\n\nQUESTIONS:\n- Does *asymptotically no regret*  simply mean that the method converges? Or that convergence is sub-linear?\n\n- Am I correct in assuming that the GPs that you are fitting when using BTS-RED are homoskedastic with a small noise level?\n\n- Is there an intuitive meaning to the variable $R^2$? Or is this a hyper-parameter to choose? I think it would be good to talk about it earlier, as it is often mentioned, but not discussed until page 5.\n\n- When doing Thompson Sampling for for BTS-RED, the variance of the GP is scaled (e.g. line 4 in algorithm 1). Is this done for theoretical guarantees or for some other reason?\n\n- I do not fully understand section 3.2.2 (Upper Bound on Noise Variance Function). If the posterior of $g$ is a GP is the result that $-\\mu_{t-1}'(x) + \\beta_t' \\sigma_{t-1}'(x)$ an upper-bound not obvious? If we talk about the general case where $\\epsilon'$ is not necessarily Gaussian, then the posterior is not necessarily a GP which could mean my confusion is just a typo in line 214.\n\nMinor formatting issues:\n- I found the figures and their font sizes to be too small and difficult to read.\n- In Theorem 3.1 $\\tau_{t-1}$ and $\\beta_t$ are defined and then not used in the Theorem\n\nLIMITATIONS:\nLimitations are mentioned in the Conclusion section. Other limitations I mentioned (e.g. lacking empirical evaluation) seem like they could be easily addressed by the authors in a single iteration of the paper. There is no obvious negative social impact.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-06-12"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces an algorithm for selecting both sampling locations and the number of replications in the context of heteroskedastic Bayesian Optimization. The authors propose to use Thompson sampling for batch candidate selection and a scheme for determining the number of replications for each element of the batch based on an effective noise variance. The authors propose three different variants of their algorithm, covering both the noiseless and noisy cases, as well as the mean-variance optimization case. They prove asymptotic regret bounds for each case (based on which they suggest the choice of effective noise variance). Finally, the paper empirically compares the proposed algorithm against baselines on synthetic and real-world problems.\n\nSTRENGTHS:\n- This is a well-motivated problem and of interest to the Bayesian Optimization research community as well as to practitioners.\n- While the basic ideas aren't novel, the specific approach taken is. It is also quite simple and easy to implement. \n- The theoretical results appear sound and are useful, especially the guidelines for how to choose the effective noise variance. The technical contributions are solid and non-trivial.\n- The paper is generally well written and the contributions are stated clearly.\n\nWEAKNESSES:\n- Theoretical results:\n    - While the results are interesting, it's not really clear to me to what extent the asymptotic rates from this approach differ from those that would be achieved by uniform sample allocation (see my questions below). This seems like a missing aspect to me that the authors should address.\n- Empirical results:\n    - It feels like the authors could have attempted harder to compare against other baselines.\n        - The comparison is only against sequential BO algorithms, which have a clear disadvantage in that they cannot, well, use batch evaluations (this includes RAHBO). While the authors acknowledge that, the obvious thing to do here would be to compare against parallel (batch) BO algorithms (using uniform budget allocation) such as qEI, qUCB, etc. that are readily available in the literature (and are implemented in many BO libraries). This would help better understand the performance of the proposed method in absolute terms (rather than solely in terms of improving over batch TS due to non-uniform allocation.\n        - Similarly, even though [34] may be heuristic, it would still be useful to compare against empirically (if that is reasonably straightforward to do).\n    - The improvements relative to a basic baseline such as simple batch TS do not strike me as particularly impressive.\n        - This may be due to the simple problems that were used. It would be helpful to see how the algorithm performs on higher dimensional problems (max dimension is 2) and what effect the dimension has on the significance of the ability to be able to allocate budget in a nonuniform fashion. \n        - Maybe these results are more impressive than I think, but if that's the case maybe the authors can try to explain this better?\n- Figures: The figures in the MT are tiny and extremely hard to read. I suggest moving some of the technical details from section 3 to the appendix and generally tighten up the writing and make some readable figures instead.\n\nQUESTIONS:\n- Asymptotic no-regret:\n    - Wouldn't a uniform sampling strategy achieve the same? Are there any improved regret bound rates that we can get from the proposed approach vis-a-vis a uniform allocation?\n    - *Note*: I am not questioning the practical value of doing non-uniform allocation, but I wonder if the bound is more interesting - especially even with a uniform allocation I would assume that in a finite space you should asymptotically achieve similar rates (log(B)/sqrt(B)) in terms of the budget as if you used the proposed strategy. The non-asymptotic regime seems more interesting). Not that this is still with heteroskedastic noise, so this is somewhat different than comments on the homoskedastic noise case the authors make in l164.\n- What if the noise is heteroskedastic in the outcome value (rather than necessarily the input)? Can this be exploited in a different way?\n- What would a more principled approach for the n_max heuristic for avoiding samples on undesirable regions look like? Could this be handled in a more principled as part of the point generation?\n- For the unknown noise case, the GP modeling the empirical noise variance may produce negative predictions. How are you dealing with this? Have you considered modeling the log-variance with a GP instead (a common approach in other similar work)?\n- Finiteness of the domain: The discretization approach (\"we assume that the domain X is finite, since extension to compact domains can be easily achieved via suitable discretizations [4].\") is fine in theory (for regret bounds etc) but can become problematic in practice, especially in higher dimensions where the number of discrete points to sample needs to grow exponentially with the dimension to get similar coverage of the domain, which renders acquisition function optimization challenging. It seems reasonably straightforward to optimzie the acquisition fucniton using gradient-based optimization, maybe the authors may want to comment on that.\n- \"it is recommended to make nmin larger in experiments where the overall noise variance is large\" <- why is that recommended? It will make it easier to estimate the local noise variance, but it's not necessarily clear that this is going to result in better optimization performance.\n- \"When the noise variance is unknown [...] we approximate sigma_max^2 by the maximum observed empirical noise variance and update our approximation after every iteration\" <- This makes a lot of sense, but it feels like something is swept a bit under the rug here. Specifically, what does this do to the regret bound, which  depends on sigma_max^2? Is it still valid (just with an unknown factor)? Or is there additional work to show this?\n- The agriculture example is a nice real-world application and a good fit for the algorithm, however, I question the number of iterations that are being evaluated here. How long is a growing cycle until the leaf area and tipburn area can be evaluated? If this is on the order of months then 100 growth cycles seem a lot...\n- The regularization parameter lambda depends on the overall horizon T (and the appendix states that this is required for the theoretical results to hold) - however, in practice T may not be know if we are interested in anytime performance of the algorithm. How would one deal with this in practice?\n- Why are you optimizing the GP hyperparameters only after every 10 iterations and not after every iteration? \n- typo l97: \"upper on\"\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper proposes a batched Bayesian optimization with heterodescadic noise and Thompson sampling. Further, it proposes an extension where not only function is minimized but also a robust variant objective which also incorporates observational noise variance. It provides some regret analysis relying on prior works and experimental comparison on real life benchmarks from agriculture.\n\nSTRENGTHS:\nOriginality\n---------\nThe use of Thompson sampling with heteroscedastic noise is novel. \n\nSignificance\n-----------\n1. Authors test their algorithm in real-world settings. \n2. I think the pattern of trying to repeat measurements is indeed omnipresent in the life sciences, and methodologies which automatically incorporate it are of benefit to practitioners. \n\nClarity & Quality\n--------\nI was able to understand the paper, however the exposition is very dense.\n\nWEAKNESSES:\nThe proof techniques used here are classical used in prior works mainly Chowbury & Gopalan 2017 and Desaultes et. al. 2014. Namely the RKHS proof of Gopalan and Batched version of it using the uncertainty sampling there. This work provides a Thompson sampling look at heteroscedastic bandits but the theoretical results are only of marginal interest to the community since they are straightforward extension of prior techniques. \n\nThe focus on Thompson sampling seems arbitrary. There are works which can address heteroscedastic as well as robust optimization but do not use Thompson sampling such as Kirschner & Krause 2018 and Makarova e.t al. 2021. Why Thompson sampling in particular is interesting remains unanswered. \n\nThe fact one performs repetitions is not theoretically motivated, but instead only practically motivated; or at least I took the liberty to make this conclusion. Information theoretically speaking, repeating the measurements might not be the best thing to do to decrease uncertainty overall. In fact probably optimizing an allocation over all points can lead to a better solution in long run (over multiple iterations). However, I think studying the problem where the repetitions are a constraint on the measurement scheme is much better motivation for this work, since life-sciences do have certain setup-costs per experiment.\n\nQUESTIONS:\n1. Why do you put heavy focus on the theory? Are the theoretical bounds important for your real-life experiments?\n2. You provide a outline how to choose R, but is this what you use in your real lab-experiments. \n3. In real experiment - do you use uncertainty sampling as in the theorems? \n\n\nI my opinion this provides limited novelty towards broader NeurIPS crowd. I think as a case study of Bayesian optimization with heteroscedasdic noise its very nice, and a paper more focused towards actual challenges with precision agriculture would be much more fitting. I think studying the problem where the repetitions are a constraint on the measurement scheme is much better motivation for this work.\n\nLIMITATIONS:\nAuthors adequately addressed the limitations and, if applicable, potential negative societal impact of their work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose three methods for Bayesian Optimization under the constraint of batch sampling and with significant heteroscedastic aleatoric uncertainty assumed. Their methods BTS-RED-Known assumes knowledge of the variance function, BTS-RED-Unknown does not assume the variance function is known and fits a GP to the negative variance function and Mean-Var-BTS-RED, also assumes no knowledge of the variance function but uses the learned variance function to enable risk averse BO by optimizing for a weighted combination of the mean function and the negative variance.\n\nThe authors prove their methods are asymptotically no-regret. The methods are tested on synthetic as well as real-world (precision agriculture and hyperparameter optimization) setups.\n\nSTRENGTHS:\n- The paper is well written and clear.\n- The BTS-RED methods are simple and intuitive, it would be easy to implement and apply them to new problems.\n- The method performs well on the limited (see below) empirical evaluation conducted.\n- The authors prove the BTS-RED methods are asymptotically no-regret.\n- The Mean-Var extension for risk averse BO is interesting.\n\nWEAKNESSES:\n- The method is developed for discrete valued domains, IMO extension to real-valued domains may not be trivial (see limitations section).\n- The experiments are all conducted on very low dimensional, discrete domains with small bounds. I would have liked to see evaluations on real-valued domains and higher dimensional problems e.g. hyperparameter tuning with 10-20 mixed type hyperparameters. Currently the empirical evaluation is quite limited as a result.\n- For the synthetic and precision agriculture experiments, the authors guarantee the ground truth function is of the heteroscedastic GP model class that their method assumes. This is not unreasonable, but given that there is only one further experiment in the paper which comes from a real-world unknown function (hyperparameter tuning experiment) this limits the robustness of the empirical evaluation as it gives the BTS-RED methods quite an advantage over competitor methods which do not have a heteroscedastic GP function representation. I would have liked to see further evaluated on more unknown ground truth functions.\n- Overall I am slightly concerned with the substantiveness of the contribution of the paper. The use of a heteroscedastic GP and Thompson sampling for BO is well established. The extension to batch sampling is interesting, but to what extent the BTS-RED methods interestingly integrate batch structure seems limited to me (see questions below). The Mean-Var extension is again interesting but fairly straightforward. All together the novelty and contribution seems somewhat limited compared to prior work e.g. [16].\n\nQUESTIONS:\n- I was surprised to see that the second GP was used to model $- \\simga^2$. Is this correct? Why is the GP not used to model $log \\simga^2$ as is more standard in the literature as it can take on values on the full real-valued range?\n- I am not sure I fully understand the implementation of non-batch BO algorithms in the experimental section. It would seem to me that a non-batch algorithm should have an advantage in that they should sample 1 point, make an observation, update the function representation and sample again. Of course this may make them infeasible to apply to problems with batch structure e.g. precision agriculture. However in the experiments these methods are performing very poorly, so I am assuming they are not implemented as above. Are you using the non-batch methods in a batch-wise fashion? I assume this would mean that all these methods would in effect simply sample the exact same point many times per batch until the budget is used up? If my understanding is correct, I am not sure this is an interesting comparison and it would be interesting to evaluate these methods as they were intended to be used as a upper bound on performance compared to methods with the batch constraint.\n- Why is there a need to compute the empricial mean over the replications (line 9, algorithm 1)? It would seem to me that adding (x, y_1), ..., (x, y_n) data points to the training data would be a better use of the data as it would explicitly weight observations with more replications more highly. It seems to me that you are losing the weighting by the number of replications be computing the empirical mean of the observations and treating all such aggregated data points as equal (line 10)?\n- Do I understand correctly that in the various BTS-RED methods, the batch structure is only accounted for in the heuristic regarding replicating the same point too many times? Otherwise sample points are selected independently, with no cross-datapoint correlations accounted for and no knowledge of the prior points in batch? This would seem like a significant weakness of the method, and it seems feasible to modify the method to account for this, see for example \"BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning\" for a similar treatment in the related active learning space.\n\nLIMITATIONS:\nThe authors address some limitations of the method e.g. the use of a heuristic to handle unused budget in the last iteration of the algorithm. However I would liked to have seen a more comprehensive discussion of other limitations:\n\n- The method is tested on discrete valued domains, it is claimed extension to the real-valued domain is feasible, however it is not clear to me that the heuristics introduced to ensure that the same point is not sampled repeatedly can be trivially extended to a real-valued domain where it would be possible to sample a very near by (essentially identical) point without adding to the count of the number of replications for the originally sampled point.\n- Heuristics are introduced to ensure the same point is not sampled too many times and the method introduces new hyperparameters which cannot all be set following apriori guidelines.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-10"}