Title: Failure-Aware Gaussian Process Optimization with Regret Bounds
Venue: NeurIPS 2023
Accepted: yes
Keywords: Gaussian process optimization

regret analysis

black-box optimization

Bayesian optimization
Abstract: Real-world optimization problems often require black-box optimization with observation failure, where we can obtain the objective function value if we succeed, otherwise, we can only obtain a fact of failure. Moreover, this failure region can be complex by several latent constraints, whose number is also unknown. For this problem, we propose a failure-aware Gaussian process upper confidence bound (F-GP-UCB), which only requires a mild assumption for the observation failure that an optimal solution lies on an interior of a feasible region. Furthermore, we show that the number of successful observations grows linearly, by which we provide the first regret upper bounds and the convergence of F-GP-UCB. We demonstrate the effectiveness of F-GP-UCB in several benchmark functions, including the simulation function motivated by material synthesis experiments.

Failure-Aware Gaussian Process Optimization with Regret Bounds

Abstract

Real-world optimization problems often require black-box optimization with observation failure, where we can obtain the objective function value if we succeed, otherwise, we can only obtain a fact of failure. Moreover, this failure region can be complex by several latent constraints, whose number is also unknown. For this problem, we propose a failure-aware Gaussian process upper confidence bound (F-GP-UCB), which only requires a mild assumption for the observation failure that an optimal solution lies on an interior of a feasible region. Furthermore, we show that the number of successful observations grows linearly, by which we provide the first regret upper bounds and the convergence of F-GP-UCB. We demonstrate the effectiveness of F-GP-UCB in several benchmark functions, including the simulation function motivated by material synthesis experiments.

1 Introduction

The effectiveness of optimization methods based on Gaussian process (GP) regression for expensiveto-evaluate black-box functions has been repeatedly shown in a wide range of real-world applications, including robotics [34, 35], experimental design [14], and hyperparameter optimization [4, 41]. On the other hand, failure of the observation itself must often be considered. For example, in the optimization of hyperparameters for a complicated physical model, the evaluation may crash for some hyperparameters. Another example is materials development, in which experimental testing of new materials can reveal that the synthesis procedure fails.

Therefore, this study considers the optimization of a black-box function f with a black-box deterministic failure function c : X → { 0 , 1 } , where X is an input space. In this study, failure of observation at the input x is represented as c ( x ) = 1 , while the success of observation is represented as c ( x ) = 0 . Then, our optimization problem can be formulated as follows:

where the observation of f can be obtained only if c ( x ) = 0 , but the observation cost is consumed even if c ( x ) = 1 . Hence, the goal is to efficiently identify the optimal point x ∗ while considering the observation failure.

One of the notable technical difficulties of problem (1) is how to handle the failure function c . To model c , existing studies [32, 2] use the GP classification (GPC) model [38] or its variants, which assume that c can be represented by a smooth latent function. However, the observation failure can

be caused by several latent constraints implicitly. In addition, since only the binary failure can be obtained, the number of latent constraints is also unknown. Therefore, modeling c with the usual GPC is hard and often unsuitable, which can degrade the optimization performance. Hence, an optimization method under a mild assumption on c is demanded. Furthermore, the existence of c makes theoretical analysis and even securing the number of successful observations difficult.

Our contribution. In this paper, we propose a novel GP-based optimization algorithm, failureaware GP upper confidence bound (F-GP-UCB), which chooses the next evaluation in an input domain except for the adaptively adjusted neighborhood of the past observation failures. Furthermore, we show that the number of successful observations grows linearly under a very mild assumption of the failure function c that the optimal solution lies on an interior of a feasible region. Then, we provide the first regret upper bound of the GP optimization problem (1), which shows that F-GP-UCB converges to the optimal solution with high probability. We demonstrate the effectiveness of the F-GP-UCB algorithm with several benchmark functions, including a heuristic simulation function motivated by materials research of quasicrystals.

Related work. In the past few decades, many GP-based optimization algorithms have been developed [37, 42, 18, 8, 53]. In addition to the standard optimization settings, various extensions have been studied, such as parallel [9], high-dimensional [54], multi-fidelity [25], and robust optimization [6].

GP-based constrained optimization [10, 11, 19, 47] has a close relation to our study. It considers the black-box optimization under an inequality constraint g ( x ) ≤ 0 , where g is also a black-box function. If the failure function in this paper is recast as c ( x ) = 1 l { g ( x ) > 0 } , the optimization problem matches that of problem (1). However, there are two crucial differences. First, in constrained optimization, observations at an input can be obtained even if the constraint is not satisfied, i.e., g ( x ) > 0 . Under the setting of this paper, no information about the objective function can be obtained for input points that result in failure. Second, in constrained optimization problems, it is generally assumed that noisy observation of g ( x ) is possible. In the setting of our study, we cannot obtain a direct observation from the latent function g .

There exist works on GP optimization that take into account failures. Lindberg and Lee [32] proposed an algorithm that combines the Expected Improvement (EI) criterion [37] with the posterior success probability of the classical GPC. Instead of GPC, Sacher et al. [39] used a support vector machinebased model. However, although c is deterministic, these classifiers assume that observation failure is essentially stochastic, i.e., the evaluations at the same input can fail and succeed. This model misspecification can degrade the optimization performance, as described in Bachoc et al. [2]. Then, Bachoc et al. [2] proposed the deterministic variant of GPC which models c as c ( x ) = 1 l { g ( x ) ≥ 0 } , where g is a latent function modeled by a GP. They also provide an EI-based strategy and prove convergence to an optimal solution. From a practical point of view, surrogate models of c navigate the optimization process efficiently when c is well represented through a smooth latent function g . However, practical applications often have too complex failure processes to model with one latent function (e.g., the failure function c depends on several latent functions.) Furthermore, it is difficult to know that such a complex failure structure exists beforehand. Finally, from a theoretical perspective, Bachoc et al. [2] give the convergence guarantee, but its rate and the regret-based analysis are not provided.

2 Preliminaries

Problem setup. Let f : X → R and c : X → { 0 , 1 } be an unknown fixed objective function and an unknown failure function, respectively, where X := [0 , 1] d is the input space. At each time step t , the user makes a selection x t ∈ X and obtains the failure label c ( x t ) . If c ( x t ) = 0 , the user proceeds to make a noisy observation y t = f ( x t ) + ϵ t where ϵ t is a noise term which is conditionally σ -sub-Gaussian. Our noise model is a mild one; examples include an arbitrary distribution over [ -σ, σ ] and a Gaussian noise with variance below σ 2 . Note that, in the case of c ( x t ) = 1 (failure), the user obtains no further information.

The user's goal is to efficiently identify the optimal solution x ∗ over the unknown feasible region S c := { x ∈ X | c ( x ) = 0 } . For convenience, we rephrase the problem (1) using S c as follows:

̸

We assume that S c = ∅ and that there exists a unique solution x ∗ . Furthermore, we define the failure region F c as F c = X \ S c .

Regret. We employ the regret to evaluate the algorithm's performance. The regret r t at step t is defined as r t = f ( x ∗ ) -f (ˆ x t ) if ˆ x t ∈ S c , otherwise r t = f ( x ∗ ) -min x ∈X f ( x ) , where ˆ x t ∈ X is the algorithm's estimated solution 1 . In our definition of r t , the algorithm is supposed to incur the worst-case regret when ˆ x t is not in S c . Similar definitions are also used in performance metrics of GP-based constrained optimization [19].

Regularity assumptions for the objective function. As a regularity assumption, we assume that f is an element of the reproducing kernel Hilbert space (RKHS) H k , corresponding to a known positive-definite kernel k : X × X → R , and has a bounded Hilbert norm ∥ f ∥ H k ≤ B . Furthermore, the kernel k is assumed to be normalized, namely ∀ x ∈ X , k ( x, x ) ≤ 1 . These are common assumptions in existing GP optimization literature [42, 51, 8, 48].

Gaussian process modeling. Our algorithm uses the modeling information of f from GP [38], using only the successful observations. First, we assume zero-mean GP with the covariance function k as the prior of f . Next, we model the generating process of the value y t queried at x t as y t = f ( x t )+ ϵ t , where ϵ t ∼ N (0 , σ 2 ) . We note that the assumptions on the GP prior on f and Gaussian noise, as stated above, are assumptions only for constructing the GP model, which can differ from the underlying true function f and the noise. Let I ( S ) t := { i ∈ { 1 , . . . , t } | c ( x i ) = 0 } be the index set of successful observations, X ( S ) t := ( x i ) i ∈I ( S ) t be the corresponding inputs, y ( S ) t := ( y i ) i ∈I ( S ) t , be the corresponding outputs, and n t := |I ( S ) t | be the number of successful observations. In addition, we also define the failure index set I ( F ) t := { i ∈ { 1 , . . . , t } | c ( x i ) = 1 } . Given X ( S ) t , y ( S ) t , the posterior distribution of f ( x ) becomes a normal distribution, whose posterior mean µ t ( x ) and posterior variance σ 2 ( x ) are given as follows:

t

Here, I n t is a n t × n t identity matrix. Furthermore, denoting the index of the j -th least element of I ( S ) t as i j , k ( x , X ( S ) t ) ∈ R n t represents a vector whose j -th element is k ( x , x i j ) . Similarly, K ( X ( S ) t ) ∈ R n t × n t represents the kernel matrix whose jk -th element is k ( x i j , x i k ) .

Lastly, we define the maximum information gain [42, 49] as a quantity representing the GP complexity. The maximum information gain γ t ; A from observing t number of data points over the set A ⊂ R d is defined as γ t ; A = max X :=( x 1 ..., x t ) ∈ A t I ( f X , y X ) , where I ( f X , y X ) := 0 . 5 ln det ( I t + σ -2 K ( X ) ) is the mutual information between the latent function values f X ∼ N ( 0 , K ( X )) and corresponding output values y X = f X + ϵ t , ϵ t ∼ N ( 0 , σ 2 I t ) of GP. In GP-based optimization, γ t ; A is often used as a quantity that characterizes the confidence bound and regret bound of f . Furthermore, for a compact convex set A , the upper bound for γ t,A has been derived for some commonly used kernels. For example, γ t ; A = O ((ln t ) d +1 ) for Gaussian kernel and γ t ; A = O ( t d/ ( ν + d ) (ln t ) 2 ν/ ( ν + d ) ) for Matérn kernel with the smoothness parameter ν > 1 / 2 [42, 49]. The following Lemma 2.1 adapts the well-known result that gives the confidence bound of f for our problem setup, which is a direct consequence of Theorem 3.11 in [1].

Lemma 2.1. Fix f ∈ H k with ∥ f ∥ H k ≤ B . Suppose the observation y t = f ( x t ) + ϵ t has a noise ϵ t that is conditionally σ -sub-Gaussian. Define { β t } t ∈ N as β 1 / 2 t = B + σ √ 2( γ t -1; X +1+ln(1 /δ )) for δ ∈ (0 , 1) . Then the following holds with probability at least 1 -δ :

Here, lcb t ( x ) and ucb t ( x ) are defined as lcb t ( x ) = µ t -1 ( x ) -β 1 / 2 t σ t -1 ( x ) and ucb t ( x ) = µ t -1 ( x ) + β 1 / 2 t σ t -1 ( x ) , respectively.

Regularity assumption for failure function. A regret upper bound cannot be obtained without any assumption on c . As an extreme example, we consider the case where x ∗ is an isolated point surrounded by F c . Under this scenario, the worst case exists in which an arbitrary algorithm can never observe x ∗ in a finite number of trials. Therefore, in order to give a convergence guarantee, at least x ∗ must be contained in a subset of the feasible region S c having a non-zero volume. In this paper, we focus on the case that x ∗ is the interior point of S c as in the following assumption.

Assumption 2.2. There exists η > 0 such that N x ∗ ; η ⊂ S c , where N x ∗ ; η := { x ∈ X | ∥ x -x ∗ ∥ ∞ < η } is an open infinity ball with a radius η centered at x ∗ 2 .

The parameter η above depends on the size of the subset of the feasible region that x ∗ belongs to and is an important quantity for theoretical analysis. Note that Assumption 2.2 is quite mild. For example, when c is defined as c ( x ) = 1 l { g ( x ) ≥ 0 } with a continuous latent function g , there exists η > 0 such that N x ∗ ; η ⊂ S c from the continuity of g . Since the analysis of [2] assumes the existence of the latent function g , which is a continuous sample path generated from GP, our assumption also subsumes their assumption. Finally, note that η is not needed for running the algorithm and is only used as a quantity that characterizes the regret bound.

3 Proposed algorithm

Our proposed algorithm F-GP-UCB is shown as pseudo-code in Algorithm 1. Roughly speaking, our F-GP-UCB searches the input domain excluding the adaptively adjusted neighborhood of past failure points based on the existing GP-UCB [42] strategy. Below, we start by describing the background of the algorithm construction.

Philosophy of algorithm construction. Since F c is unknown, the algorithm needs to appropriately avoid the failure observation in F c while balancing the trade-off between exploration and exploitation in a way that guarantees convergence. The difficulty of this problem lies in the fact that there exist cases where the small feasible region containing x ∗ is surrounded by the past failure observations and is isolated as shown in the left plot of Fig. 1. Since the algorithm is unable to exclude the possibility that an arbitrary small feasible region exists between failure points only from Assumption 2.2, the algorithm should be constructed so that it simultaneously satisfies the following two points: the failure observation should be avoided; and the inputs evaluated by the algorithm should eventually cover the arbitrarily close area of the past failure point.

F-GP-UCB satisfies the above requirements by adaptively controlling the GP-UCB search region by eliminating the neighborhood of the past failure points, and shrinking the eliminated region as step increase. Next, we describe below the details of and ideas behind each step.

Selection of x t . At each step t , the F-GP-UCB algorithm firstly computes the seach region X t as

̸

Here, θ t and b : R + → R + are the scale parameter and monotonically decreasing function, respectively. The parameter θ t is controlled by the algorithm at every step so that it decreases monotonically with respect to t from its initial value θ 0 given by the user. It plays the role of guaranteeing that the search space X t satisfies X t = ∅ . The function b is defined by the user before running the algorithm.

By using X t , the F-GP-UCB algorithm chooses x t as

The middle and right plots in Fig. 1 show an example behavior of the F-GP-UCB algorithm. By using the monotonically decreasing θ t b ( t ) to control the search space, the algorithm's behavior can be qualitatively described as follows. First, during the early phase where t is small, θ t b ( t ) is large, corresponding to choosing an unexplored point for observation while avoiding a large neighborhood of the past failure points where there is a high possibility of failure. Then as t increases and the

Iteration 5

Iteration 10

remaining unexplored space shrinks, it is expected that the algorithm's behavior will become more aggressive in considering the possibility that the region near the observed failure points contains a feasible region that may be difficult to identify.

The choice of θ 0 and b affects the performance of F-GP-UCB. In Sec. 4, we show the convergence of the regret under an appropriate choice of b . In Sec. 5, the practical choices of θ 0 and b are discussed.

Shrinking θ t . In the case where X t = ∅ , x t is not defined. We ensure such cases do not occur by controlling the scale parameter θ t . In our algorithm, the main role of θ t is to guarantee that x t is well-defined as follows. Specifically, before the step t starts, θ t is computed by halving the previous scale parameter θ t -1 until the union of all the neighborhoods does not fully cover X . This procedure requires that we solve the set covering problem to find the cover of X with the neighborhoods of the past failure points. Unfortunately, this problem is known to be NP complete [20]. In Sec. 5, we provide a practical approach to this problem. On the other hand, note that our analysis in Sec. 4 assumes that this procedure can be computed exactly.

Algorithm estimated solution. The estimated solution of the F-GP-UCB algorithm ˆ x t at step t is defined as follows:

The definition of the estimated solution based on the observed points and lcb i is often used in exisiting literature [6, 26, 21]. While the estimated solution of the proposed method resembles those in the

Algorithm 1 The F-GP-UCB algorithm

̸

Input:

θ

0

∈

(0

,

1)

,

b

:

R

+

→

R

+

,

{

β

t

}

t

∈

N

+

.

1: Initialize GP prior and set I ( S ) 0 = I ( F ) 0 = ∅ .

2:

for

t

= 1

to

T

do

3: ˜ θ t ← θ t - 1 .

4: while X ⊂ ⋃ i ∈I ( F ) t N x i ; ˜ θ t b ( t ) do

5:

˜

θ

t

˜

θ

t

/

2

.

6: end while

7: θ t ← ˜ θ t and define X t as in (6).

8:

Choose

x

t

= argmax

x

∈X

t

ucb

t

(

x

)

.

9:

if

c

(

x

t

) = 0

then

10:

Observe

y

t

=

f

(

x

t

) +

ϵ

t

and update GP.

11:

I

(

S

)

t

←I

(

S

)

t

-

1

∪ {

t

}

,

I

(

F

)

t

←I

(

F

)

t

-

1

.

12:

else

13:

I

(

S

)

t

←I

(

S

)

t

-

1

,

I

(

F

)

t

←I

(

F

)

t

-

1

∪ {

t

}

.

14:

end if

15:

end for

16:

Calculate

x

ˆ

T

as in (8) if

I

(

S

)

T

=

∅

.

literature, it should be noted that the maximum of lcb i is computed over I ( S ) t . Note ˆ x t is not defined if all the points up to t are failures (i.e., I ( S ) t = ∅ ). In Sec. 4, it is shown that there always exists ˆ x T for a sufficiently large step size T . Finally, although the estimated solution (8) is useful for discussing the theoretical performance, for practical purposes, ˆ x t = x ˆ t where ˆ t = argmax i ∈I ( S ) t lcb t ( x i ) is often used instead [6, 21]. We use this modified definition of ˆ x t in the benchmark experiments in Sec. 6.

4 Theoretical analysis

The results of the theoretical analysis are discussed in this section. The complete proofs of the theorems and lemmas are described in Appendix A.

Key of regret analysis. To analyze the regret, we start by evaluating the growth rate of the number of successful observations n t in the algorithm. By noting that θ t b ( t ) is monotonically decreasing, we can find that the failure points of F-GP-UCB are separated by at least a distance θ t b ( t ) apart from each other. Therefore, n t can be evaluated via θ t b ( t ) -packing number of F c . The following Lemma 4.1 is based on the standard argument for the packing number, which gives the lower bound for the number of successful observations n t .

Lemma 4.1. When running Algorithm 1, n t ≥ t -P ( X , ∥ · ∥ ∞ , θ t b ( t ) / 4) holds for any t ∈ N + , where P ( X , ∥ · ∥ ∞ , ϵ ) denotes the ϵ -packing number over the set X with respect to the norm ∥ · ∥ ∞ .

Using the above Lemma 4.1, designing the choice of b so that t -P ( X , ∥ · ∥ ∞ , θ t b ( t ) / 4) = Θ( t ) holds leads to n t = Ω( t ) , which means that the number of successful observations can be secured at the same rate as in the standard GP optimization setting. By using the fact P ( X , ∥ · ∥ ∞ , ϵ ) = ⌈ 1 /ϵ ⌉ d (Lemma A.4), the choice of b ( t ) = o ( t -1 /d ) is sufficient for n t = Ω( t ) 3 . While our result holds for any b satisfying b ( t ) = o ( t -1 /d ) , we give results for b satisfying b ( t ) = t -α with α ∈ (0 , 1 /d ) in this section for simplicity. In Appendix A, the result for general X and b is also provided.

Regret bound. The following Theorem 4.2 gives the regret bound of the F-GP-UCB algorithm.

Theorem 4.2. Fix δ ∈ (0 , 1) , α ∈ (0 , d -1 ) , X = [0 , 1] d , and f ∈ H k such that ∥ f ∥ H k ≤ B . Let β 1 / 2 t = B + σ √ 2( γ t -1; X +1+ln(1 /δ )) and b ( t ) = t -α . In addition, suppose that there exists η > 0 such that N x ∗ ; η ⊂ S c , where N x ∗ ; η = { x ∈ X | ∥ x -x ∗ ∥ ∞ < η } . When applying Algorithm 1 under the above conditions, the estimated solution ˆ x T defined in (8) exists for all T ≥ ˜ T s := max { (10 /θ d ) 1 / (1 -dα ) , 2 } +1 , where θ = min { θ 0 , η/ 2 } . Moreover, the following holds with probability at least 1 -δ :

In the analysis above, η is interpreted as the complexity of the failure function c , and θ corresponds to the lower bound on θ t while running the algorithm. The constants ˜ T s and ˜ T ∗ are important quantities that characterize the regret and can be respectively interpreted as follows.

The constant ˜ T s is the upper bound on the number of steps until the first successful observation.

The constant ˜ T ∗ is the upper bound on the number of steps needed until the search space X t of the algorithm always contains the optimal solution x ∗ .

The regret bound in Theorem 4.2 can be intuitively seen as evaluating the algorithm execution in terms of two stages along the time axis. The first term of (9) represents the regret generated when the algorithm's search space X t does not contain x ∗ . Specifically, the worst-case regret incurred during this phase is bounded by 2 B using the fact that sup x ∈X | f ( x ) | ≤ B from the kernel normalization

condition. The second term is the regret incurred during the process of identifying x ∗ after X t includes x ∗ and is similar to the regret bound of standard GP-UCB. The only difference is that, in our setup, the observations of GP are only on S c , so the maximum information gain is defined over S c .

By noting that γ T ; S c ⊂ γ T ; X in the second term and using the known results concerning the upper bound on γ T ; X , the more explicit form of regret upper bound can be obtained on a perkernel basis. For example, in the case of the Gaussian kernel, we have that γ T ; X = O ((ln T ) d +1 ) and r T = O ( T -1 B ˜ T ∗ + T -1 / 2 (ln T ) ( d +1) / 2 ( B + σ √ (ln T ) d +1 +ln(1 /δ ) )) . Thus, indeed we obtain r T → 0 ( as T → ∞ ) . The F-GP-UCB algorithm is guaranteed to converge when the cumulative regret for ordinary GP-UCB (corresponding to the second term in (9)) becomes sub-linear under the chosen setup and the kernel 4 .

The case where x ∗ exists on the boundary of S c . The case where x ∗ exists on the boundary S c is not covered in Theorem 4.2, which is the case that often appears in a real-world application. If undesirable Lipschitz-style dependencies are allowed to emerge in the regret upper bound, our F-GP-UCB algorithm is also guaranteed to have convergence in such cases (Theorem B.2). The details of the result and discussion about its limitation are in Appendix B.

5 Practical considerations

In this section, we discuss several issues that arise in practical situations and provide their solutions. Due to the space limitation, we only give brief descriptions. The details are described in Appendix E.

̸

Computation of θ t . The computation of θ t requires us to know whether the union of all the neighborhoods of the failure points covers X . A simple solution might be to partition X into a sufficiently fine grid by subdividing each axis and look for a feasible solution in each grid cell by brute force. However, as the step size t and dimension d become large, the number of grid cells grows rapidly. Furthermore, even a very fine grid will not be able to detect the case where a feasible solution exists on the cell boundary. Another heuristic approach is to use a constrained optimization solver which does not require a feasible solution as part of the initial points (e.g., penalty function method and augmented Lagrangian method [17]) and try to solve the problem (7) by using θ t -1 . In case that the solver is unable to find a feasible solution, it is decided that ˜ X t = ∅ , multiply θ t -1 by 1 / 2 , then we try to solve problem (7) again. This is our chosen approach. A drawback of this approach is that, depending on the scale of θ t , the constrained optimization problem must be solved many times. We show that, by performing an appropriate preprocessing, a feasible solution can be obtained within two rounds of constrained optimization in the worst case. We give the details in Appendix E.1.

Selection of b . As stated in Sec. 4, the F-GP-UCB algorithm is valid for an arbitrary monotonically decreasing function b such that t -P ( X , ∥·∥ ∞ , θ t b ( t ) / 4) = Θ( t ) . However, if the rate is slower than 1 /t α ( α ∈ (0 , 1 /d )) , for example, b ( t ) = 1 / ln( t +1) , then the dominant term of our regret upper bound can become O (1 / ln T ) (See Theorem B.3 in Appendix B and Corollary F.1 in Appendix F). This intuitively means that an exponentially large sample is needed to obtain an arbitrarily small regret and is not the desirable behavior. In this paper, we use b in the form of b ( t ) = 1 /t α with α = 1 / (2 d ) as a practical choice. (Additional discussion about α is given in Appendix E.2.)

Adaptive tuning of θ 0 . Although the choice of θ 0 does not affect the no-regret guarantee, it has a large impact on the practical behavior when the number of steps is small. For example, if θ 0 is set to a large value when the failure region F c forms a relatively small region, then there is a risk of excessively avoiding the feasible region that should be searched. It is practically difficult to find a desirable choice of θ 0 as it depends on some conditions that are unknown to the user, such as the failure region F c . In order to alleviate this issue while retaining the theoretical convergence, we use the strategy to adaptively choose θ t based on the posterior standard deviation σ t -1 ( x t ) of the observed points. We note that such a strategy using σ t -1 ( x t ) to adaptively select an unknown parameter is also developed in [54] to set the unknown kernel hyperparameters for ordinary GP optimization.

√

Algorithm 4 in Appendix E.3 shows the pseudo-code of our modified strategy. In Algorithm 4, the user specifies the possible range for θ t ∈ [ θ min , θ max ] , the threshold for the posterior standard deviation h σ > 0 , the threshold for the step number q ∈ N + , and a scaling factor w ∈ (0 , 1) . As the initial value, we set θ 0 = θ max. After that, if the observed point is lower than the posterior standard deviation for q consecutive number of times, θ t is multiplied by w to lower its value, but no less than the minimum value θ min. Intuitively, suppose the posterior standard deviation is repeatedly excessively small. In that case, there exists a possibility that θ 0 was set to an excessively large value, which may lead to the unwanted exclusion of the feasible region from the search space. In this procedure, by setting θ 0 to a large value of θ max at the start, we expect it to get adjusted towards an appropriate neighborhood scale. In our simulation experiments, we set w = 0 . 75 , h σ = 0 . 02 , q = 3 , θ min = 0 . 0001 , and θ max = 0 . 5 . We emphasize that this procedure does not affect our convergence guarantee (Theorems F.2 in Appendix E).

Lastly, the pseudo-code, including all the considerations provided in this section, is described in Algorithm 5 of Appendix E.

6 Numerical experiments

In this section, we show the performance of F-GP-UCB through numerical experiments. The detailed settings of the experiments and additional results can be found in Appendices G and H, respectively. First, for comparison, we use EI [37] and GP-UCB [42] as the baseline algorithms which do not consider failures. In each of these algorithms, the GP model is constructed using only the successful observations; the failure observations are not used. We also compare F-GP-UCB with the EFI-GPCEP and EFI-GPC-Sign algorithms that leverage GPC models [2]. EFI-GPC-EP is the algorithm that uses the classic GPC model with probit likelihood, and posterior approximation is based on Expectation Propagation (EP). EFI-GPC-Sign uses a variant of GPC proposed in [2]. Both EFI-GPCEP and EFI-GPC-Sign choose the next input based on the posterior success probability of GPC and the EI value of f as in [2] (Details are in Appendix H).

In EFI-GPC-EP and EFI-GPC-Sign, as the prior for the latent function of GPC, we use zero-mean GP with Gaussian kernel k c ( x , y ) := σ 2 c exp( -∥ x -y ∥ 2 / (2 l 2 c )) . Before the experiment, we fix the kernel parameters of GPC σ c and l c by marginal likelihood maximization using observations at a randomly generated Sobol sequence. All the kernel hyperparameters of the GPC fixed by this procedure are given in Appendix H. Furthermore, it is well-known that the theoretical choice of β t in GP-UCB is excessively conservative. We set β t = 2ln(2 t ) as in [24] for GP-UCB and F-GP-UCB. The other parameters in F-GP-UCB are fixed as described in Sec. 5. For the evaluation of the algorithms, we confirm the behavior of the regret at each step r t = f ( x ∗ ) -f (ˆ x t ) . We note that ˆ x t is the estimated solution ˆ x t = x ˆ t where ˆ t = argmax i ∈I ( S ) lcb t ( x i ) , which is slightly

t

modified from the theoretical value as discussed in Sec. 3 If we get that I ( S ) t = ∅ , then we set r t = f ( x ∗ ) -min x ∈X f ( x ) . This definition is the same as the utility gap metric for constrained GP optimization [19]. In each experiment, we report the average performance of 20 numbers of optimization trials with different seeds. In each trial, we generate one point uniformly at random over X and use it as the initial point. Lastly, we add an artificial noise ϵ t ∼ N (0 , σ 2 ) with σ 2 = 0 . 0001 to the observation of f . We also fix the noise variance hyperparameter of the GP model for f to 0 . 0001 .

Synthetic experiments with the Branin function. We perform synthetic benchmark experiments with the Branin function whose input space is scaled to [0 , 1] 2 . The failure function used in these experiments is shown in Fig. 2. This failure function has a feasible region on the upper right side which is easily identifiable. In addition, the failure function also contains multiple small isolated feasible regions, one of which contains the optimal solution. We use the Gaussian kernel k ( x , y ) := σ 2 f exp( -∥ x -y ∥ 2 2 / (2 l 2 f )) to construct a GP model of f . The parameters σ f and l f are fixed beforehand by marginal likelihood maximization over a Sobol sequence of 1024 points. The subsequent experiments also use the Gaussian kernel; and the parameters are fixed as described above.

Figure 2 shows the result. Under this benchmark setting, F-GP-UCB has a better performance. We also give the detailed behavior of F-GP-UCB in Appendix G. We note that, in EI and GP-UCB, the regret stops decreasing in the early stage. This is because these algorithms get stuck on the same failure point.

0.0

0.2

0.4

0.6

0.8

1.0

Input 1

0.0

0.2

0.4

0.6

0.8

1.0

Input 2

Branin test problem

Optimal solution

0

50

100

150

200

250

Iteration

10

1

10 0

10 1

10 2

Regret

Branin test function

GP-EI

GP-UCB

EFI-GPC-EP

EFI-GPC-Sign

F-GP-UCB

GP-EI

GP-UCB

EFI-GPC-EP

EFI-GPC-Sign

F-GP-UCB

0

20

40

60

80

100 120 140

Iteration

10

4

10

3

10

2

10

1

10 0

Regret

Gardner test function

0

50

100

150

200

250

Iteration

10

2

10

1

10 0

Regret

Hartmann test function

0

25

50

75

100 125 150 175 200

Iteration

10

2

Regret

Quasicrystal simulation function

Test functions for constrained optimization. We adopt the two benchmark settings which are used in existing GP-based constrained optimization literature. Gardner [10] is a 2D test problem whose objective and constraint functions are defined as a combination of sine and cosine functions. Hartmann [29] is a 3D test problem 5 whose feasible region becomes a unit hypersphere. To adapt the constrained problem to our settings, the feasible region in each benchmark is defined as the success region, while the rest is defined as the failure region. Fig. 3 shows the result. Within the compared algorithms, F-GP-UCB has the best performance for the Gardner function, while EFI-GPC-Sign has the best performance for the Hartmann function. This can be understood from the fact that the feasible region of the Hartmann function, which is a unit hypersphere, and can be modeled easily by GPC. However, F-GP-UCB performs better compared to EFI-GPC-EP, which uses a classic GPC model. Moreover, the regret of F-GP-UCB continues to decrease as the number of iterations increases.

Numerical experiments with quasicrystals. We perform a numerical experiment involving quasicrystals in the Al-Cu-Mn system. We consider the optimization of the phonon thermal conductivity for quasicrystals. In this setting, the input space is the relative composition of the three elements, restricted to be around the feasible region of quasicrystal formation. The feasible region is the composition values that form quasicrystals based on data [16]. Outside of this region, quasicrystals do not form and so we consider it as a failure region, as the material property could not be probed with methods designed for quasicrystals. For the objective function, we use an empirical formula for the phonon thermal conductivity which we extract from [46]. The details of the setting are described in

Appendix H.2. In Fig. 3, the right plot shows the result of the numerical experiments for quasicrystals. It shows that F-GP-UCB performs the best among the compared algorithms.

7 Conclusions

In this paper, we propose a novel GP-based optimization algorithm in the presence of unknown failure regions in the search space. Our algorithm only requires a very mild assumption of the failure function that the optimal solution lies on an interior of a feasible region. We show that our algorithm achieves a convergence with high probability and provides the first regret upper bound by appropriately adjusting the search space. Its effectiveness is verified empirically through numerical experiments, including the heuristic simulation experiment motivated by the material research of quasicrystals.

References

Yasin Abbasi-Yadkori. Online learning for linearly parametrized control problems . PhD thesis, University of Alberta, 2013.

François Bachoc, Céline Helbert, and Victor Picheny. Gaussian process optimization with failures: classification and convergence proof. Journal of Global Optimization , 78(3):483-506, 2020.

Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham, Andrew G Wilson, and Eytan Bakshy. Botorch: A framework for efficient Monte-Carlo Bayesian optimization. Advances in Neural Information Processing Systems , 33:21524-21538, 2020.

James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for hyperparameter optimization. In Advances in Neural Information Processing Systems , volume 24, 2011.

Felix Berkenkamp, Andreas Krause, and Angela P Schoellig. Bayesian optimization with safety constraints: safe and automatic parameter tuning in robotics. Machine Learning , pages 1-35, 2021.

Ilija Bogunovic, Jonathan Scarlett, Stefanie Jegelka, and Volkan Cevher. Adversarially robust optimization with Gaussian processes. Advances in Neural Information Processing Systems , 31, 2018.

Romain Camilleri, Kevin Jamieson, and Julian Katz-Samuels. High-dimensional experimental design and kernel bandits. In International Conference on Machine Learning , pages 1227-1237. PMLR, 2021.

Sayak Ray Chowdhury and Aditya Gopalan. On kernelized multi-armed bandits. In Proceedings of the 34th International Conference on Machine Learning , volume 70, pages 844-853, 2017.

Thomas Desautels, Andreas Krause, and Joel W. Burdick. Parallelizing exploration-exploitation tradeoffs in Gaussian process bandit optimization. Journal of Machine Learning Research , 15 (119):4053-4103, 2014.

Jacob R Gardner, Matt J Kusner, Zhixiang Eddie Xu, Kilian Q Weinberger, and John P Cunningham. Bayesian optimization with inequality constraints. Proceedings of the 31st International Conference on Machine Learning , 32(2):937-945, 22-24 Jun 2014.

Michael A. Gelbart, Jasper Snoek, and Ryan P. Adams. Bayesian optimization with unknown constraints. In Proceedings of the Thirtieth Conference on Uncertainty in Artificial Intelligence , UAI'14, page 250-259, Arlington, Virginia, USA, 2014. AUAI Press. ISBN 9780974903910.

Alan Genz. Numerical computation of multivariate normal probabilities. Journal of Computational and Graphical Statistics , 1(2):141-149, 1992.

Susana Gomez and Alejandro Velasco Levy. The tunnelling method for solving the constrained global optimization problem with several non-connected feasible regions. In Numerical Analysis , pages 34-47. Springer, 1982.

Javier González, Joseph Longworth, David C James, and Neil D Lawrence. Bayesian optimization for synthetic gene design. arXiv preprint arXiv:1505.01627 , 2015.

GPy. GPy: A Gaussian process framework in python. http://github.com/SheffieldML/ GPy , since 2012.

Benjamin Grushko and SB Mi. Al-rich region of Al-Cu-Mn. Journal of Alloys and Compounds , 688:957-963, 2016.

PC Haarhoff and JD Buys. A new method for the optimization of a nonlinear function subject to nonlinear constraints. The Computer Journal , 13(2):178-184, 1970.

Philipp Hennig and Christian J. Schuler. Entropy search for information-efficient global optimization. Journal of Machine Learning Research , 13(57):1809-1837, 2012.

José Miguel Hernández-Lobato, Michael A. Gelbart, Ryan P. Adams, Matthew W. Hoffman, and Zoubin Ghahramani. A general framework for constrained Bayesian optimization using information-based search. Journal of Machine Learning Research , 17(1):5549-5601, 2016.

Jörg Hoffmann and Sebastian Kupferschmid. A covering problem for hypercubes. In IJCAI , pages 1523-1524, 2005.

Shogo Iwazaki, Yu Inatsu, and Ichiro Takeuchi. Mean-variance analysis in Bayesian optimization under uncertainty. In International Conference on Artificial Intelligence and Statistics , pages 973-981. PMLR, 2021.

Steven G. Johnson. The NLopt nonlinear-optimization package , 2011. URL http:// ab-initio.mit.edu/nlopt .

Donald R Jones, Cary D Perttunen, and Bruce E Stuckman. Lipschitzian optimization without the Lipschitz constant. Journal of Optimization Theory and Applications , 79(1):157-181, 1993.

Kirthevasan Kandasamy, Jeff Schneider, and Barnabas Poczos. High dimensional Bayesian optimisation and bandits via additive models. In Francis Bach and David Blei, editors, Proceedings of the 32nd International Conference on Machine Learning , volume 37 of Proceedings of Machine Learning Research , pages 295-304, Lille, France, 07-09 Jul 2015.

Kirthevasan Kandasamy, Gautam Dasarathy, Junier Oliva, Jeff Schneider, and Barnabas Poczos. Multi-fidelity Gaussian process bandit optimisation. Journal of Artificial Intelligence Research , 66:151-196, 2019.

Johannes Kirschner, Ilija Bogunovic, Stefanie Jegelka, and Andreas Krause. Distributionally robust Bayesian optimization. In International Conference on Artificial Intelligence and Statistics , pages 2174-2184. PMLR, 2020.

Shunya Kusakawa, Shion Takeno, Yu Inatsu, Kentaro Kutsukake, Shogo Iwazaki, Takashi Nakano, Toru Ujihara, Masayuki Karasuyama, and Ichiro Takeuchi. Bayesian Optimization for Cascade-Type Multistage Processes. Neural Computation , 34(12):2408-2431, 11 2022. ISSN 0899-7667. doi: 10.1162/neco_a_01550.

Madison Lee, Shubhanshu Shekhar, and Tara Javidi. Multi-scale zero-order optimization of smooth functions in an rkhs. In 2022 IEEE International Symposium on Information Theory (ISIT) , pages 288-293. IEEE, 2022.

Benjamin Letham, Brian Karrer, Guilherme Ottoni, and Eytan Bakshy. Constrained Bayesian optimization with noisy experiments. Bayesian Analysis , 14(2):495-519, 2019.

Zihan Li and Jonathan Scarlett. Gaussian process bandit optimization with few batches. In International Conference on Artificial Intelligence and Statistics , pages 92-107. PMLR, 2022.

Zihan Li and Jonathan Scarlett. Regret bounds for noise-free cascaded kernelized bandits. CoRR , abs/2211.05430, 2022.

David V Lindberg and Herbert KH Lee. Optimization under constraints by applying an asymmetric entropy measure. Journal of Computational and Graphical Statistics , 24(2):379393, 2015.

Chang Liu, Erina Fujita, Yukari Katsura, Yuki Inada, Asuka Ishikawa, Ryuji Tamura, Kaoru Kimura, and Ryo Yoshida. Machine learning to predict quasicrystals from chemical compositions. Advanced Materials , 33(36):2102507, 2021.

Daniel J Lizotte, Tao Wang, Michael H Bowling, Dale Schuurmans, et al. Automatic gait optimization with Gaussian process regression. In IJCAI , volume 7, pages 944-949, 2007.

Ruben Martinez-Cantin, Nando de Freitas, Arnaud Doucet, and José A Castellanos. Active policy learning for robot planning and exploration under uncertainty. In Robotics: Science and systems , volume 3, pages 321-328, 2007.

Thomas P Minka. Expectation propagation for approximate Bayesian inference. In Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence , pages 362-369, 2001.

Jonas Moˇ ckus. On Bayesian methods for seeking the extremum. In Optimization Techniques IFIP Technical Conference , pages 400-404, 1975.

Carl Edward Rasmussen and Christopher K. I. Williams. Gaussian Processes for Machine Learning . The MIT Press, 11 2005. ISBN 9780262256834. doi: 10.7551/mitpress/3206.001. 0001.

Matthieu Sacher, Régis Duvigneau, Olivier Le Maitre, Mathieu Durand, Elisa Berrini, Frédéric Hauville, and Jacques-André Astolfi. A classification approach to efficient global optimization in presence of non-computable domains. Structural and Multidisciplinary Optimization , 58(4): 1537-1557, 2018.

Sudeep Salgia, Sattar Vakili, and Qing Zhao. A domain-shrinking based bayesian optimization algorithm with order-optimal regret performance. Advances in Neural Information Processing Systems , 34:28836-28847, 2021.

Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical Bayesian optimization of machine learning algorithms. In Advances in Neural Information Processing Systems , volume 25, 2012.

Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proceedings of the 27th International Conference on Machine Learning (ICML-10) , pages 1015-1022, 2010.

Yanan Sui, Alkis Gotovos, Joel Burdick, and Andreas Krause. Safe exploration for optimization with Gaussian processes. In Proceedings of the 32nd International Conference on Machine Learning , volume 37 of Proceedings of Machine Learning Research , pages 997-1005, Lille, France, 07-09 Jul 2015.

Yanan Sui, Vincent Zhuang, Joel Burdick, and Yisong Yue. Stagewise safe Bayesian optimization with Gaussian processes. In International conference on machine learning , pages 4781-4789. PMLR, 2018.

Krister Svanberg. A class of globally convergent optimization methods based on conservative convex separable approximations. SIAM Journal on Optimization , 12(2):555-573, 2002.

Yoshiki Takagiwa, Ryota Maeda, Satoshi Ohhashi, and An-Pang Tsai. Reduction of thermal conductivity for icosahedral Al-Cu-Fe quasicrystal through heavy element substitution. Materials , 14(18):5238, 2021.

Shion Takeno, Tomoyuki Tamura, Kazuki Shitara, and Masayuki Karasuyama. Sequential and parallel constrained max-value entropy search via information lower bound. In Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , pages 20960-20986, 17-23 Jul 2022.

Sattar Vakili, Nacime Bouziani, Sepehr Jalali, Alberto Bernacchia, and Da-shan Shiu. Optimal order simple regret for gaussian process bandits. Advances in Neural Information Processing Systems , 34:21202-21215, 2021.

Sattar Vakili, Kia Khezeli, and Victor Picheny. On information gain and regret bounds in Gaussian process bandits. In International Conference on Artificial Intelligence and Statistics , pages 82-90. PMLR, 2021.

Sattar Vakili, Jonathan Scarlett, and Tara Javidi. Open problem: Tight online confidence intervals for RKHS elements. In Conference on Learning Theory , pages 4647-4652. PMLR, 2021.

Michal Valko, Nathan Korda, Rémi Munos, Ilias Flaounas, and Nello Cristianini. Finite-time analysis of kernelised contextual bandits. In Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence , pages 654-663, 2013.

Roman Vershynin. High-dimensional probability: An introduction with applications in data science , volume 47. Cambridge University Press, 2018.

Zi Wang and Stefanie Jegelka. Max-value entropy search for efficient Bayesian optimization. In Doina Precup and Yee Whye Teh, editors, Proceedings of the 34th International Conference on Machine Learning , volume 70 of Proceedings of Machine Learning Research , pages 3627-3635, 06-11 Aug 2017.

Ziyu Wang, Frank Hutter, Masrour Zoghi, David Matheson, and Nando De Feitas. Bayesian optimization in a billion dimensions via random embeddings. Journal of Artificial Intelligence Research , 55:361-387, 2016.

Peer reviews:
comments: SUMMARY:
1. A failure-aware gaussian process optimization algorithm is proposed. A key difference from existing GP constrained optimization is that here we cannot obtain a direct observation from the latent failure function.
2. The algorithm's regret bound is presented.
3. Some practical issues are discussed.
4. Numerical experiments are conducted, the proposed algorithm shows superiority on some benchmarks.

STRENGTHS:
1. Problem formulation is novel.
2. Theoretical derivation is solid
3. Practical issues are considered and numerical experiments show its potential.

WEAKNESSES:
1. Learning feasible set without knowing failure function is a key feature of paper's problem formulation. The NP-complete set covering problem is assumed to be fully solved in theoretical analysis while numerical experiments adopt a heuristic approximation algorithm.  It seems there is a gap between theoretical part and numerical part. 
2. Algorithm doesn't show dominant superiority on all all numerical benchmarks.

QUESTIONS:
Can we have a more "rigorous" theoretical analysis of the "real" algorithm(replacing set covering part by its heuristic approximation) and still derive the regret bound?

LIMITATIONS:
No potential negative social impact is seem.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-05

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies Bayesian optimization in the setting where observing the unknown function might fail, for instance due to numerical simulator returning NaN. The authors propose a UCB-type acquisition function for this setting, which is based on first restricting the domain to a feasible regions where failures are less likely to occur, and then applying standard UCB-based acquisition function constrained to that region. The authors present a bound on the simple regret in this setting, and perform some experiments which consider only low-dimensional examples but show good performance compared to baselines.\n\nEDIT: I've updated my score in light of the rebuttal.\n\nSTRENGTHS:\nThe paper's topic is important and of key relevance for practical Bayesian optimization, where observation failure is a common issue which is mostly addressed using ad-hoc methods. They have clearly described their setting and distinguished it from others, and everything is properly cast into a well-defined mathematical problem, which is clearly described.\n\nThe introduction is well-written and does a good job of introducing the reader to both the setting and the authors' approach, and gives due credit to prior work.\n\nFigure 1 is conceptually very nice and provides a good illustration that makes it easy for the reader to see early on what is going on: the authors' algorithm splits the state space into a region where observations are likely to work, and another region where they are unlikely to work. One request: **please make the fonts bigger to match the font size of the text**, so readers with not the best eyesight can actually see everything.\n\nThe authors do a good job at focusing on their part of the contribution, and don't take up lots of space stating obvious corollaries of their result, but at the same time also do a good job at stating what else can be done, for instance by combining their analysis with prior work such as RBF specific bounds in the setting without failures.\n\nOverall, I think this is a good paper with various fixable flaws described below, so I don't have too many additional comments.\n\nWEAKNESSES:\nDetails of the proposed algorithm are unclear. From my reading of the paper, I understand that what you do is define a \"failure-safe space\" consisting of the full space minus a set of infinity-balls around points where failure occurred. Then, at certain points in time whose definition I still don't fully grasp, you cut the radius of the balls by half, so that points which were previously excluded from consideration because they were too close to a failure are now considered again. Is this correct? If so, this needs to be stated much more explicitly and much more clearly, using an intuitive description in addition to equations or notation-heavy pseudocode. If not, please correct my understanding.\n* In particular, please provide a more intuitive explanation of what \\theta_t is doing, and in what cases it gets cut by half - I am still confused how when this happens\n\nExperiments\n* Unless I have missed something, the highest dimension problem used in experiments is three. This is very limiting compared to much of the literature and much smaller than many problems of practical interest\n* The quasicrystal experiment is very light on details, in spite of being a really cool illustration. This section deserves significantly more text on both the setup and results so that readers can understand and contextualize what is going on. The fact that this problem is, unless I am mistaken, three-dimensional, could also have been stated more explicitly\n\nIn the analysis, the regret bound presented seems fairly specific, since it is talking only about simple rather than cumulative regret, and has a fairly specific way through which the point returned is selected. In bandits at least this kind of setting is certainly out there, but not something I see in every paper, so I'm a bit unsure as to whether the ideas are general or very specific to this setting.\n* As a result, I'm curious: how strongly does the theoretical analysis and empirical performance depend on the specific way through which the point returned is selected?\n\nWriting: the paper tends to go back and from between parts that were very clear and well-written, and parts where I was confused. Part of this is due to typos and grammar, which is fixable. However, there are other times where grammar is not the issue, and there are many points where the paper would be much better if it provided helpful hints to the reader to orient their reading in the right way to avoid them becoming confused. Detailed presentation comments below:\n* The paper has slightly too many acronyms, for instance GPC, which is not an important enough concept in this paper to make it into an acronym and should instead be spelled out\n* There's slightly too much discussion on prior work, which takes up space that is needed elsewhere - I'd prefer to see this part tightened up\n* When I first read this, I was confused about definition of regret, namely why is r_t = f(x^*) - min_{x\\in X} f(x) not equal to zero, but then I realized this is because the authors are interested in reward maximization (bandit style) rather than cost minimization (online learning style). A line which saves the reader time by clarifying this should be added.\n* Aesthetic: curly brace notation for the posterior is really ugly - consider using parentheses\n* **Please number all equations, not just the ones you think are important.** The convention of only numbering important equations is archaic, outdated, and harmful to readers. Numbering everything prevents readers from having to ask about \"the second unnumbered equation on page 2\", and then the person who was asked is confused whether it's the second equation from the top or from the bottom. Doing so will significantly improve the paper.\n* Why should the logdet be called the maximum information gain? If you're going to use these terms, you should connect this with mutual information or some kind of related quantity. This choice of terminology should be explained rather than just parachuted in.\n* Since the notion of \"interior point\" is introduced in the definition directly below it, maybe the sentence should include \"introduced below\" or similar.\n* Assumption 2.2: please add a textual note that this is an infinity-ball, not a Euclidean ball, so that someone who misses the infinity symbol doesn't get confused\n* Proposed algorithm could be replaced with a more informative section title\n* No need to introduce packing numbers with too much detail, the audience of this paper will know about that\n\nQUESTIONS:\nLine 94: what do you mean by \"uses the modeling information of f\"? I didn't understand this sentence\n\nI'm confused about whether or not you assume you know the reproducing kernel of the RKHS the unknown function lies in explicitly, or not. Does the kernel of the unknown function need to match the kernel used for the GP prior? I think you assume it's known, which is reasonable - please let me know if that's correct and please make this more explicit in the text.\n\nTheorem 4.2: should there be a \"with probability 1-\\delta\" quantifier for \"the estimated solution exists ..\"? What stops the algorithm from getting arbitrarily unlucky and obtaining no data at arbitrarily large times? Does this have to do with the assumption that simulator failure is deterministic?\n\nTypos and English grammar fixes below:\n* Line 40: \"converges the optimal\" -> \"converges to the optimal\"\n* Line 47: what does it mean \"has a close relation to this study\"?\n* Line 52: \"under the settings\" -> \"under the setting\"\n* Line 54: \"that a noisy observation\" -> \"that noisy observation\"\n* Line 58: \"modifying the settings of our study\" -> unsure what this sentence means\n* Line 60: \"that takes into account failures\" -> \"that take into account failures\"\n* Line 63: \"that the observation failure\" -> \"that observation failure\"\n* Line 65: \"as described in [2]\" -> \"as described in Bachor et al. [2]\"\n* Line 70: \"one latent function. (e.g. ...\" -> \"one latent function (e.g. ...\"\n* Line 78: \"which is a conditional\" -> \"which is conditionally\"\n* Line 117: \"each of its observation\" -> \"each of its observations\"\n* Line 122: \"The regret upper bound cannot be obtained\" -> \"A regret upper bound cannot be obtained\"\n* Line 247: \"Practical consideration\" -> \"Practical considerations\"\n\nLIMITATIONS:\n*Almost* adequately - the main thing that is missing is a more comprehensive discussion on the dependence of their proposed methods on the dimension of the unknown function's domain.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2023-07-06"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper extends the gaussian process UCB algorithm to the case where some observations are failures. i.e. They solve for the case where x^* = argmax_{x in X} {f(x) such that c(x) =0}. Here c(x) = 1 may be thought of as an indicator of failure. The crucial part is to adaptively ignore the neighbourhood of the failure point to choose the next evaluation point. The paper does a number of experiments to show that their algorithm works with respect to the claims made.\n\nSTRENGTHS:\n- Excellent numerical experiment on quasicrystals. This was quite novel to consider the Al-Cu-Mn crystal system.\n- The idea of applications of GP-UCB to account for failures of observation was quite novel.\n- The notation of failure as a simple function c() was well formulated.\n\nWEAKNESSES:\n- The model setup is slightly weak. For example, has consideration been given to stochastic c(·)? How easy is it to extend to unbounded domain beyond [0,1]^d?\n- The definition of r_t says that when \\hat{x}_t is not observable, r_t is f(x^*) - min_{x in X} f(x). This is just 0. Shouldn't regret be based on an output of the algorithm? If non-observable, the algorithm can simply output an element from observed data that it is sure is observable. Why are you defining regret to be 0 in such a case?\n- There could be improvements in paper clarity. For example:\n\n    > Please could you clean up the notations for kernel function k? Since it plays a central role in the theorems, it would be useful to setup the relationship between k(·) and the optimization function f(·) a little more cleanly.\n\n- The authors claim that \"Then, we provide the first regret upper bound of the GP optimization problem (1)...\" But from my understanding, this well cited paper from 2009 provides an upperbound for GP-UCB (see Theorem 1)? https://arxiv.org/pdf/0912.3995.pdf\n\nQUESTIONS:\n- What is scaling of regret with dimension? Though your experiments indicate poor scaling with d, your regret expression doesn't explicitly show the dependence?\n- \\hat{T}^* = \\theta_0/\\eta)^{1/alpha} + 1, and regret is less thant B(\\hat{T}^* -1). Why add and subtract 1? What is the order dependence on T?\n- Typically people speak about cumulative, or simple regret. You have a theorem for regret r_t at time t. Can your theorem be written as a cumulative regret guarantee?\n- What are your assumptions on C() that helps you to ignore the neighbourhood of a failure point? For example, adversarially, I could give c() to be a delta function at an arbitary number of points whose neighbourhood contains the optimal function. There are strong assumptions on f(), but the assumptions on c() were not clear to me.\n\nLIMITATIONS:\n- Seems like a minor improvement over non-failure aware GP based optimization Algorithms. \n- If you disregard failures in GP-UCB, does it not converge? If so, why is the regret on that algorithm in your experiments never below 1 (what is the function c you have considered)?\n- Seems like scaling with dimension d is quite poor? Does random restart help in such cases?", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-11"}

comments: SUMMARY:
This paper investigates GPBO under a robust setting, where the failure of observations is taken into account in the overall budget.  An adaptive algorithm based on GP-UCB has been proposed to address this problem, and both theoretical and practical behavior have been analyzed and verified.

STRENGTHS:
Overall, this is a well-written paper with comprehensive analysis of the proposed algorithm. The empirical studies also demonstrate that the proposed algorithm is effective for at least low-dimensional problems.

WEAKNESSES:
The empirical behaviour of the proposed algorithm is only verified for dimensions less than 3, and it would be nicer to see more practical settings.  In such cases, a vanilla GP-UCB might work even better than a specially designed algorithm.  Thus, It might be interesting to explore the trade-offs in practice between the two approaches.

Minor:
line 154 "computes"

QUESTIONS:
To my knowledge, greedy elimination algorithms based on maximizing posterior variance usually achieve better theoratical convergence rates than UCB algorithms when it comes to robutsness considerations.  Do you have any idea that in the failure setting, a similar algorithm, say by replacing (5) with $x_t = \arg \max \sigma_t(x)$, would work better or worse?

LIMITATIONS:
NA
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2023-07-24