Title: Active Set Ordering
Venue: NeurIPS 2024
Accepted: yes
Keywords: active learning

Bayesian optimization

top-k set

contour line
Abstract: In this paper, we formalize the active set ordering problem, which involves actively discovering a set of inputs based on their orderings determined by expensive evaluations of a blackbox function. We then propose the mean prediction (MP) algorithm and theoretically analyze it  in terms of the regret of predicted pairwise orderings between inputs. Notably, as a special case of this framework, we can cast Bayesian optimization as an active set ordering problem by recognizing that maximizers can be identified solely by comparison rather than by precisely estimating the function evaluations. As a result, we are able to construct the popular Gaussian process upper confidence bound (GP-UCB) algorithm through the lens of ordering with several nuanced insights. We empirically validate the performance of our proposed solution using various synthetic functions and real-world datasets.

Active Set Ordering

Abstract

In this paper, we formalize the active set ordering problem, which involves actively discovering a set of inputs based on their orderings determined by expensive evaluations of a blackbox function. We then propose the mean prediction (MP) algorithm and theoretically analyze it in terms of the regret of predicted pairwise orderings between inputs. Notably, as a special case of this framework, we can cast Bayesian optimization as an active set ordering problem by recognizing that maximizers can be identified solely by comparison rather than by precisely estimating the function evaluations. As a result, we are able to construct the popular Gaussian process upper confidence bound (GP-UCB) algorithm through the lens of ordering with several nuanced insights. We empirically validate the performance of our proposed solution using various synthetic functions and real-world datasets.

1 Introduction

In real-world applications, we often encounter the problem of estimating an unknown function, known as a blackbox function , (i.e., those without closed-form expressions or derivatives) using their expensive and noisy evaluations. Under these circumstances, an efficient sequential process of evaluating the function is desired. On one extreme, experimental design (ED) aims to estimate the function in its entire input domain, e.g., by decreasing the uncertainty of the function globally in Bayesian ED [4, 18]. On the other extreme, the renowned Bayesian optimization (BO) targets inputs with the extreme function values such as the maximizers and the minimizers [2, 6, 7].

While the connection between ED and BO is studied in the classic work of [19], we still lack a problem formulation that strikes a balance between the prohibitively expensive process of estimating the entire function globally in ED and the lack of information about the function away from extreme locations in BO. One may consider a related problem, called level set estimation (LSE), which focuses on estimating inputs with function evaluations above or below a given (or implicit) threshold [1, 3, 8, 14]. However, without domain knowledge of the blackbox function, it is easy to set a threshold that leads to undesirably large or small level sets.

Let us consider an environmental monitoring problem of estimating a chemical concentration in a field. The blackbox function is the mapping from locations of the field to the chemical concentration measurement. It can be of a greater interest to estimate the maximizers, the minimizers, the topk locations (with the highest chemical concentration) and the bottomk locations. On one hand, these estimates provide more information about the blackbox function than just the maximizers or minimizers in BO. On the other hand, they may require less resource (i.e., evaluations of the blackbox function) than estimating the entire function in ED. Besides, as the topk locations consist of exactly k locations in the field, it circumvents the issue of undesirably large or small level sets in LSE.

Our main contribution in this paper is to formulate the above challenge and resolve it with a theoretically grounded solution. Specifically, we propose the active set ordering problem to capture the above scenario (Sec. 2.1). It aims to estimate subsets of the input domain that are defined based on pairwise comparisons/orderings between the blackbox function evaluations. 1 These subsets include the maximizers, the minimizers, and the topk inputs with the highest function evaluations. Like Bayesian ED, BO, and LSE, we adopt the pool-based active learning setting [18] in constructing a solution that sequentially selects a sampling input from the domain at each iteration. The knowledge from observing function evaluations at the sampling inputs helps predicting the subsets of interest and directs the algorithm to select the next sampling input. To facilitate the presentation of our method, we begin with the building block of our ordering-based problem: pairwise comparison/ordering between function evaluations in Sec. 3. Specifically, we propose a new kind of regret to quantify the loss of a pairwise ordering (Sec. 3.1), a prediction of the topk inputs based on only the posterior mean (Sec. 3.2), and a sampling strategy that is equipped with a theoretical performance guarantee for the proposed prediction (Sec. 3.3). Subsequently, these concepts of the regret, the prediction, and the sampling strategy are extended to orderings between sets, which ultimately addresses the active set ordering problem in Sec. 4. Notably, the regret simplifies to the well-known regret in BO (Remark 4.1). Hence, we recover both the theoretical analysis and the GP-UCB algorithm [19] when considering a special case of our problem setting (Remark 4.5). In Sec. 5, we empirically validate the performance of our solution using several synthetic functions and real-world datasets.

2 Preliminaries and Problem Statement

2.1 Topk set

Adopting an assumption in existing level set estimation (LSE) works [1, 8], we consider a blackbox function f : X → R where the domain X is a finite set of n elements in R d . Let S c ≜ X \ S denote the complement of any subset S ⊂ X . In this paper, the ordering between inputs are determined with respect to their corresponding blackbox function evaluations. Hence, we use the term 'the ordering between x and x ′ ' and 'the ordering between f ( x ) and f ( x ′ ) ' interchangeably.

Definition 2.1 (Topk set) . The topk set , denoted as S ( k ) , is the set of k inputs with the highest function evaluations. Specifically, |S ( k ) | = k and ∀ x ∈ S ( k ) , ∀ x ′ ∈ S c ( k ) , f ( x ) ≥ f ( x ′ ) .

In this work, we propose the active set ordering problem to estimate the topk set S ( k ) of a blackbox function f by efficiently gathering noisy function evaluations in a sequential manner. Furthermore, it includes the Bayesian optimization (BO) problem when k = 1 because the top1 set S (1) contains a maximizer of f .

2.2 Gaussian Process

The noisy function evaluation mentioned in the previous section is denoted as y ( x ) ≜ f ( x ) + ϵ ( x ) where the noise ϵ ( x ) ∼ N (0 , σ 2 n ) is a Gaussian random variable with a known (or estimated) variance σ 2 n . To obtain the posterior distribution of the unknown function f given these noisy evaluations, we model f using a Gaussian process (GP), that is, every subset of { f ( x ) } x ∈X follows a multivariate Gaussian distribution [16]. A GP is fully specified by its prior mean and its kernel k x , x ′ ≜ cov ( f ( x ) , f ( x ′ )) which measures the covariance between function values. Let D t denote the set of sampling inputs in the first t -1 iterations. Then, given y D t ≜ ( y ( x )) x ∈D t , the predictive distribution of any function evaluation f ( x ) follows a Gaussian distribution with the following mean and variance:

where k t ( x ) ≜ ( k ( x , x ′ )) x ′ ∈D t , K t ≜ ( k ( x , x ′ )) x , x ′ ∈D t , and I is the identity matrix [16].

Assuming f belongs to a reproducing kernel Hilbert space with its norm bounded by B > 0 , due to [5], we have the following confidence bound of f ( x ) . 2

Lemma 2.2. Pick δ ∈ (0 , 1) and set β t = ( B + σ n √ 2( γ t -1 +1+log1 /δ )) 2 . Then, the following event happens with probability of at least 1 -δ ,

where l t ( x ) ≜ µ t ( x ) -β 1 / 2 t σ t ( x ) , u t ( x ) ≜ µ t ( x ) + β 1 / 2 t σ t ( x ) , and γ t -1 ≜ max A ⊂X : | A | = t -1 I ( y A ; f A ) is the maximum information gain of f A ≜ { f ( x ) } x ∈ A through observing y A over all subsets A ⊂ X of size | A | = t -1 .

To ease notational clutter, we denote the above confidence interval of f ( x ) as C t ( x ) ≜ [ l t ( x ) , u t ( x )] and its length as |C t ( x ) | ≜ u t ( x ) -l t ( x ) .

3 Active Pairwise Ordering: n = 2

̸

From Definition 2.1, pairwise orderings (or pairwise comparisons) are the building blocks of our active set ordering problem. Hence, to facilitate the exposition of the key ideas, let us begin with a simplistic setting where the input domain X consists of only n = 2 inputs, i.e., X = { ˜ x , ˜ x ′ } and f (˜ x ) = f (˜ x ′ ) . The problem is to determine the top1 set S (1) , i.e., the maximizer of f (equivalently, the minimizer of f ). In essence, the goal is to check if f (˜ x ) > f (˜ x ′ ) by strategically collecting noisy evaluations y (˜ x ) and y (˜ x ′ ) . In particular, at iteration t , the algorithm proposes a sampling input x t ∈ X to obtain a noisy evaluation y ( x t ) . Then, the GP posterior distribution of f is updated and used to construct a predicted ordering between ˜ x and ˜ x ′ (i.e., the ordering between f (˜ x ) and f (˜ x ′ ) ). The problem boils down to the strategy of selecting the sampling input x t such that a performance metric of the predicted ordering between f (˜ x ) and f (˜ x ′ ) is satisfactory. In the next section, we introduce a regret definition to serve as a performance metric.

3.1 Regret

Let us denote the (unknown) true ordering between ˜ x and ˜ x ′ according to the evaluations of the blackbox function f as π ∗ :

π ∗ (˜ x , ˜ x ′ ) ≜ ✶ f (˜ x ) ≥ f (˜ x ′ ) (1) where the indicator function ✶ f (˜ x ) ≥ f (˜ x ′ ) = 1 if f (˜ x ) ≥ f (˜ x ′ ) and 0 otherwise. For any ordering π : { (˜ x , ˜ x ′ ) } → { 0 , 1 } , we define the following regret of π (˜ x , ˜ x ′ ) :

̸

In particular, r π (˜ x , ˜ x ′ )=1 = max(0 , f (˜ x ′ ) -f (˜ x )) and r π (˜ x , ˜ x ′ )=0 = max(0 , f (˜ x ) -f (˜ x ′ )) . The rationale is to ensure poor performance leads to large regret: The regret is | f (˜ x ) -f (˜ x ′ ) | (which increases as the gap between f (˜ x ) and f (˜ x ′ ) increases) if the ordering π does not align with the true ordering, i.e., π (˜ x , ˜ x ′ ) = π ∗ (˜ x , ˜ x ′ ) . On the contrary, the regret is 0 (i.e., the best performance) if π (˜ x , ˜ x ′ ) = π ∗ (˜ x , ˜ x ′ ) .

However, the blackbox function f renders the evaluation of r π (˜ x , ˜ x ′ ) impossible. Thus, we resort to relying on the GP posterior distribution of f at iteration t to construct an upper bound of the above regret in the following lemma (proof in Appendix A).

Lemma 3.1. For all t ≥ 1 , let us define

Then, ρ ( t ) π (˜ x , ˜ x ′ ) is an upper confidence bound of the regret r π (˜ x , ˜ x ′ ) , i.e.,

where δ is as defined in Lemma 2.2.

3.2 Prediction

Definition 3.2 (Predicted pairwise ordering π µ t ) . Given the above upper bound ρ ( t ) π (˜ x , ˜ x ′ ) of the regret, we would like to make a prediction π µ t that minimizes ρ ( t ) π (˜ x , ˜ x ′ ) . Therefore, π µ t is defined as follows

As a result, the upper confidence bound of the regret in (3) is minimized at π (˜ x , ˜ x ′ ) = π µ t (˜ x , ˜ x ′ ) and its minimum value is

We note that the upper confidence bound of the regret can be interpreted as a measure of the approximation quality or the uncertainty reduction as discussed in the following two remarks.

Remark 3.3 (Approximation quality) . Since ρ ( t ) π µt (˜ x , ˜ x ′ ) ≥ r π µt (˜ x , ˜ x ′ ) for all t ≥ 1 with probability of at least 1 -δ , the regret incurred by the ordering π µ t cannot exceed the worst-case regret ρ ( t ) π µt (˜ x , ˜ x ′ ) as shown in Fig. 1a. Hence, if | f (˜ x ) -f (˜ x ′ ) | > ρ ( t ) π µt (˜ x , ˜ x ′ ) , π µ t (˜ x , ˜ x ′ ) is the true ordering, i.e., π µ t (˜ x , ˜ x ′ ) = π ∗ (˜ x , ˜ x ′ ) , with probability of at least 1 -δ .

Remark 3.4 (Minimum uncertainty reduction) . In Fig. 1b, one can interpret ρ ( t ) π µt (˜ x , ˜ x ′ ) as the minimum amount that the confidence intervals C t (˜ x ) and C t (˜ x ′ ) (representing the uncertainty) reduce so that r π µt (˜ x , ˜ x ′ ) = 0 with probability of at least 1 -δ .

Moreover, the prediction π µ t can be obtained using only the GP posterior mean (proof in Appendix B), which explains the name of our approach: mean prediction (MP) and the notation π µ t .

Lemma 3.5 (Mean prediction) . The predicted ordering π µ t defined in (4) can be determined from the GP posterior mean

(a) ρ ( t ) π µt (˜ x , ˜ x ′ ) is the worst-case regret happening when f (˜ x ) = l t (˜ x ) and f (˜ x ′ ) = u t (˜ x ′ ) .

(b) r π µt (˜ x , ˜ x ′ ) = 0 when ( u t , l t ) are refined to ( u ′ t , l ′ t ) following an observation, i.e., the reduction in the uncertainty represented as the sum of the two red dashed segments is at least ρ ( t ) π µt (˜ x , ˜ x ′ ) .

3.3 Sampling Strategy

Given the regret in Sec. 3.1 and the predicted ordering π µ t (˜ x , ˜ x ′ ) in Sec. 3.2, we would like to select a sampling input x t ∈ X = { ˜ x , ˜ x ′ } such that the regret r π µt (˜ x , ˜ x ′ ) of the predicted ordering π µ t (˜ x , ˜ x ′ ) reduces quickly.

While r π µt (˜ x , ˜ x ′ ) is unknown, it is bounded by ρ ( t ) π µt (˜ x , ˜ x ′ ) with probability of at least 1 -δ . Hence, to reduce r π µt (˜ x , ˜ x ′ ) , we aim to reduce ρ ( t ) π µt (˜ x , ˜ x ′ ) . It is also noted that observing y ( x t ) decreases the confidence interval |C t ( x t ) | . Hence, to induce the reduction in the regret r π µt (˜ x , ˜ x ′ ) through observing y ( x t ) , we select x t such that its confidence interval |C t ( x t ) | ≥ ρ ( t ) π µt (˜ x , ˜ x ′ ) (which guarantees that |C t ( x t ) | ≥ r π µt (˜ x , ˜ x ′ ) with probability of at least 1 -δ ). For instance, choosing x t = ˜ x in the left plot of Fig. 1a satisfies this condition, but it does not in the right plot of Fig. 1a. In the following lemma, we show that the following 4 choices of the sampling input satisfy the proposed condition (see Appendix C).

Lemma 3.6. Let Q t ≜ { ˜ x ▽ ˜ x ′ , ˜ x △ ˜ x ′ , ˜ x ∨ ˜ x ′ , ˜ x ∧ ˜ x ′ } denote a set 3 of inputs at iteration t where

For any x t ∈ Q t , |C t ( x t ) | ≥ ρ ( t ) π µt (˜ x , ˜ x ′ ) .

Theorem 3.7. By sampling the input x t following Lemma 3.6, we obtain the following regret bound

where β T , γ T , and δ are as defined in Lemma 2.2.

Remark 3.8 (Sublinear cumulative regret) . If γ T is sublinear, our average cumulative regret is sublinear. This requirement is similar to most BO and LSE algorithms. It is noted that γ T is sublinear for many popular kernels. For instance, γ T = O ((log T ) d +1 ) for the squared exponential (SE) kernel as discussed in [19]. In this case, our cumulative regret bound R T ≤ O ∗ ( √ T (log T ) 2 d ) is the same as that of GP-UCB [19] (where O ∗ ( · ) denotes asymptotic expressions up to dimension-independent logarithmic factors and is the dimension of the input).

We defer the pseudocode to the next section when n ≥ 2 .

4 Active Set Ordering: n ≥ 2

In this section, we utilize the results in Sec. 3 to present the mean prediction (MP) algorithm for the active set ordering problem with n ≥ 2 .

4.1 Regret

When X consists of n > 2 inputs, there are multiple pairwise orderings between inputs in X . We overload the ordering notation π ∗ of the true pairwise ordering between 2 inputs in (1) to the ordering between 2 sets as follows: for any subsets X 0 ⊂ X and X 1 ⊂ X c 0 ,

It is noted that π ∗ ( X 0 , X 1 ) remains undefined if the two cases above are not satisfied. However, this situation does not arise in our solution. We define the regret of a set ordering (i.e., multiple pairwise orderings) as the maximum regret of all pairwise orderings:

where r π ( x , x ′ ) is defined in (2) and X 0 ×X 1 is the Cartesian product of X 0 and X 1 , i.e.,

Remark 4.1 . It is noted that r π ( X 0 , X 1 ) coincides with the well-known regret in BO when we consider the problem of predicting a maximizer of f . In particularly, predicting ˆ x ∗ as a maximizer of f is equivalent to predicting the set ordering π ( { ˆ x ∗ } , X \ { ˆ x ∗ } ) = 1 . Its regret is r π ( { ˆ x ∗ } , X\{ ˆ x ∗ } )=1 = max x ∈X f ( x ) -f (ˆ x ∗ ) as shown in Appendix E.1.

Following the upper confidence bound of the regret of pairwise orderings in (3), we show in Appendix F that with probability of at least 1 -δ , for all t ≥ 1 and for all subsets X 0 ⊂ X , X 1 ⊂ X c 0 , r π ( X 0 , X 1 ) ≤ ρ ( t ) π ( X 0 , X 1 ) where

4.2 Prediction

In this section, we generalize the prediction in Sec. 3.2 to set orderings. From Lemma 3.5, there is no contradiction in the pairwise orderings π µ t ( x , x ′ ) (defined in (4)) for all { x , x ′ } ⊂ X . In other words, the transitivity property holds for the binary relation π µ t as shown in Appendix G.

Definition 4.2 (Predicted topk set S µ t ( k ) ) . Let S µ t ( k ) be a subset of X such that

where the set ordering π µ t ( S µ t ( k ) , S c µ t ( k )) is obtained by substituting π ∗ with the pairwise ordering π µ t (see Definition 3.2) in (7). From Lemma 3.5, S µ t ( k ) is basically the set of k inputs with the highest GP posterior mean values.

As π µ t ( S µ t ( k ) , S c µ t ( k )) = 1 implies that π µ t ( x , x ′ ) = 1 for all ( x , x ′ ) ∈ S µ t ( k ) × S c µ t ( k ) , the upper confidence bound of the regret is

4.3 Sampling Strategy

Like in Sec. 3.3, our key idea is to select x t such that the length |C t ( x t ) | of its confidence interval bounds ρ ( t ) π µt ( S µt ( k ) , S c µt ( k )) (in (10)). Since ρ ( t ) π µt ( S µt ( k ) , S c µt ( k )) is the maximum upper confidence bound of the regret of all pairwise orderings involved in defining S µ t ( k ) , we first determine the input pair (¯ x t , ¯ x ′ t ) that incurs the maximum upper confidence bound of the regret. This is also the input pair that π µ t most likely makes a mistake, following the intuition from [12].

It is noted that (¯ x t , ¯ x ′ t ) is constructed from an input in the predicted topk set S µ t ( k ) and an input in its complement S c µ t ( k ) . As the estimation of the topk set improves, we expect these 2 inputs to be at both sides of the boundary of the topk set: inside the topk set vs. outside the topk set.

Then, extended from Lemma 3.6, the following lemma shows that the above desirable property is satisfied by choosing the sampling input x t from any inputs in the set ¯ Q t ≜ { ¯ x t ▽ ¯ x ′ t , ¯ x t △ ¯ x ′ t , ¯ x t ∨ ¯ x ′ t , ¯ x t ∧ ¯ x ′ t } (defined in Lemma 3.6).

The proof is shown in Appendix H. As a result, the cumulative regret incurred by choosing x t in Lemma 4.3 is bounded in the following theorem (proof in Appendix I).

Theorem 4.4. By sampling x t following Lemma 4.3, we obtain the following cumulative regret bound

where β T , γ T , and δ are as defined in Lemma 2.2.

We call the algorithm that makes prediction using the GP posterior mean and selects the sampling input x t following Lemma 4.3 the mean prediction (MP) algorithm. Its pseudocode is shown in Algorithm 1. Theorem 4.4 indicates that MP incurs a sublinear cumulative regret for several commonly used kernels with sublinear γ T [19].

Remark 4.5 (Bayesian optimization as an active set ordering problem with k = 1 ) . We show in Appendix E.2 that when k = 1 , ¯ x t ▽ ¯ x ′ t ∈ argmax x ∈X u t ( x ) , which is the sampling input in the GP-UCB algorithm [19]. Additionally, as discussed in Sec. 4.1, the regret r π ( S (1) , S c (1)) is the wellknown regret in BO. Hence, we recover both the GP-UCB algorithm and its regret bound when k = 1 (although we consider the regret of the prediction rather than that of the sampling input). Moreover, this new construction of GP-UCB leads to some subtle insights. Firstly, while the GP posterior mean has been used in computing the inference regret of entropy search methods [9, 21], there has not been any theoretical justification for using the posterior mean. In contrast, the theoretical analysis in our work justifies the use of the maximizer of the GP posterior mean as an estimate of the maximizer of f . Secondly, by predicting the maximizer using the GP posterior mean, ¯ x ▽ ¯ x ′ is not the only sampling input that achieves a sublinear cumulative regret. In fact, there are other choices of the sampling input as shown in Lemma 4.3. Similarly, we note that the LCB algorithm to find the minimizer of a blackbox function can be recovered by setting k = n -1 and x t = ¯ x △ ¯ x ′ .

Remark 4.6 (Lower bound of active set ordering problem) . Let the lower bound of the active set ordering problem be the lower bound of the cumulative regret of the worst-case problem instance over all possible values of k . Then, it should be at least as large as the lower bound of the special case where k = 1 , which is the BO problem according to Remark 4.5. Furthermore, BO has known lower bounds for several common kernels, e.g., for the SE kernel, the lower bound of the cumulative regret is Ω( √ T (log T ) d/ 2 ) [17]. Hence, the lower bound of the active set ordering problem is at least Ω( √ T (log T ) d/ 2 ) . Additionally, similar to Remark 3.8, the cumulative regret of our solution in Theorem 4.4 is bounded by R T ≤ O ∗ ( √ T (log T ) 2 d ) . Hence, it matches the lower bound up to the replacement of d/ 2 by 2 d + O (1) .

Remark 4.7 . Updating the GP posterior belief incurs O ( |D t | 3 + n |D t | 2 ) (including O ( |D t | 3 ) for training and O ( n |D t | 2 ) for prediction). Given the GP posterior belief, Algorithm 1 involves the following 2 major steps. First, in line 3 of Algorithm 1, it takes O ( n log k ) to find the topk inputs S µ t ( k ) by using a max heap of size k and scanning through the GP posterior mean of all n inputs. Second, in line 4 of Algorithm 1, it takes O ( k ( n -k )) to scan through the elements in S µ t ( k ) × S c µ t ( k ) . Therefore, an iteration of Algorithm 1 takes O ( |D t | 3 + n |D t | 2 + n log k + k ( n -k )) .

Remark 4.8 (Active multiple set ordering) . Let us consider the problem of estimating m topk sets: S ( k 1 ) , S ( k 2 ) , . . . , S ( k m ) simultaneously (motivated in Sec. 1). This problem is analogous to finding k contour lines of a blackbox function, where each contour line represents the boundary between S ( k i ) and its complement S c ( k i ) . To solve this problem, we define the following input pair

In other words, we aim to reduce the maximum regret incurred by the predicted pairwise orderings in all m topk sets. Given ( ¯ ¯ x t , ¯ ¯ x ′ t ) in (12), MP proceeds by sampling the input x t according to Lemma 4.3, i.e., x t ∈ { ¯ ¯ x t ▽ ¯ ¯ x ′ t , ¯ ¯ x t △ ¯ ¯ x ′ t , ¯ ¯ x t ∨ ¯ ¯ x ′ t , ¯ ¯ x t ∧ ¯ ¯ x ′ t } . The approach is elaborated in Appendix J.

1.0

1.5

Blackbox function

GP mean

Upper bound

Lower bound

Correct Predicted T op-k

Incorrect Predicted T op-k

Missing Top-k

Current Observation

Past Observation

(a) MP:

x

t

= ¯

x

t

∧

¯

x

′

t

.

(b) Var:

x

t

= argmax

x

∈X

σ

2

t

(

x

)

.

Iteration40

Iteration40

1.5

O

1.0

0.5

0.0

0.5

1

-1.0

-1.5

-2.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

0.2

0.4

0.6

0.8

1.0

0.0

We also note that active multiple set ordering is able to find both maximizers S (1) and minimizers S c ( n -1) simultaneously, a problem has not been studied in GP-UCB [19].

5 Experiments

5.1 Active Set Ordering

1.5 1.0 In this section, we validate the empirical performance of our MP algorithm with different choices of the sampling input in Lemma 4.3: ¯ x t ▽ ¯ x ′ t , ¯ x t △ ¯ x ′ t , ¯ x t ∨ ¯ x ′ t , and ¯ x t ∧ ¯ x ′ t by comparing with 2 baselines: an uncertainty sampling approach, called Var , that selects the sampling input with the highest GP posterior variance, i.e., x t ∈ argmax x ∈X σ 2 t ( x ) , and a baseline, called Rand , that selects the sampling input at random. The regret r π µt ( S µt ( k ) , S c µt ( k )) is used to measure the performance of each algorithm, i.e., the prediction of the topk set consists of the k inputs with the highest GP posterior mean.

0.0 2.0 To begin with, we visualize sampling inputs and the accuracy of S µ t (20) that come from our MP algorithm with x t = ¯ x t ∧ ¯ x ′ t and the Var algorithm in Fig. 2. In Fig. 2a, the histogram shows that the sampling inputs are at the boundary of S (20) . This is highly desirable as it is challenging to decide if an input at the boundary belongs to S (20) . Similarly, the input pair in (11) also consists of inputs around this boundary (depicted as vertical orange lines). On the other hand, precisely estimating the function evaluations of inputs far from the boundary, e.g., inputs around x = 0 . 85 (in S (20) ) and inputs around x = 0 . 1 (not part of S (20) ) is unnecessary. We observe that the uncertainty of the GP posterior distribution at these inputs is high in Fig. 2a. Hence, our MP algorithm is able to efficiently concentrate its sampling budget on important inputs at the boundary of the topk set. Interestingly, this boundary serves as a contour line of the blackbox function, indicating that our solution could potentially be applied to estimate the contour line by specifying the proportion of the input domain where function evaluations exceed this contour. Regarding the Var algorithm (i.e., uncertainty sampling) in Fig. 2b, the histogram shows that sampling inputs are distributed evenly across the input domain. It is because Var aims to reduce the uncertainty of the function evaluation throughout the input domain without considering the current predicted S µ t (20) . For example, it is inefficient to select sampling inputs far away from the boundary of S (20) . It is observed that the estimation of function evaluations at the boundary of S (20) using Var is more uncertain than that using the MP algorithm given the same number of sampling inputs. This results in erroneously predicting certain inputs in S (20) (depicted as red dots) and overlooking several inputs in S (20) (depicted as black dots). 4

We numerically report the performance using the proposed regret r π µt ( S µt ( k ) , S c µt ( k )) . The experiments are conducted on 4 synthetic functions: a function sampled from a GP, Branin-Hoo function, Goldstein-Price function with a noise of σ n = 0 . 1 , and Hartmann-6D function with a nosie of σ n = 0 . 01 [20]. For the first three synthetic functions, the input domain is discretized into a set of 100 points, whereas for the Hartmann-6D function, it is discretized into a set of 1000 points. Motivated by environmental monitoring problems, we generate 3 active set ordering problems that

0.5

0.2

estimate the top5 set using the dataset of NO3 concentration in the Lake Zurich (downloaded from https://wldb.ilec.or.jp/Lake/EUR-06/datalist ), the dataset of the phosphorus concentration in the Brooms Barn [22], and the dataset of the humidity in the Intel Lab (downloaded from https://db.csail.mit.edu/labdata/labdata.html ). The environment field is discretized into a set of 100 locations in the experiments with the NO3 and humidity datasets and 400 locations in the experiment with the phosphorus dataset. The experiments are repeated 15 times to account for the randomness in the generation of the observations. Further details are provided in Appendix K. The average and the standard error of the regret are shown in Figs. 3s:a-g. There is not any significant difference in the performance of MP with different sampling inputs in Lemma 4.3. Nevertheless, the MP algorithm with any choice of the sampling input in ¯ Q t outperforms the 2 baselines by converging to lower regret.

̄

10 -1

10 0

10 1

10 2

Regret

Rand

Var

x

t

∧

x

0

t

x

t

∨

x

0

t

x

t

▵

x

0

t

x

t

▿

x

0

t

0

20

40

Iteration

10 -2

10 0

10 2

Regret

0

50

100

Iteration

10 -1

10 0

10 1

10 2

Regret

0

50

100

Iteration

10 -2

10 -1

10 0

10 1

10 2

Regret

0

50

100

Iteration

10 0

3×10 -1

4×10 -1

6×10 -1

Regret

(s:a) GP sample

(s:c) Goldstein-Price

(s:d) Hartmann-6D

0

50

100

Iteration

10 -1

10 0

10 1

10 2

Regret

0

25

50

75

100

Iteration

10 1

10 2

10 3

Regret

0

20

40

Iteration

10 1

10 2

Regret

0

20

40

Iteration

10 -1

10 0

10 1

10 2

Regret

(s:e) NO3

(s:f) Phosphorus

(s:g) Humidity

(m:a) GP sample

0

50

100

Iteration

10 -1

10 0

10 1

10 2

Regret

0

25

50

75

100

Iteration

10 1

10 2

Regret

0

50

100

150

Iteration

10 0

4×10 -1

6×10 -1

Regret

0

25

50

75

100

Iteration

10 0

10 1

10 2

Regret

(m:c) Goldstein-Price

(m:d) Hartmann-6D

(m:e) NO3

0

25

50

75

100

Iteration

10 3

10 4

Regret

0

20

40

Iteration

10 1

10 2

10 3

Regret

(m:f) Phosphorus

(m:g) Humidity

̄

̄

̄

̄

̄

̄

̄

0

10

10 -2

5.2 Active Multiple Set Ordering

Iteration

To empirically validate the performance of our MP algorithm in solving the problem of estimating multiple topk sets, we consider the problem of estimating S (1) (i.e., maximizers), S (10) , and S (20) simultaneously, i.e., k 1 = 1 , k 2 = 10 , k 3 = 20 in Remark 4.8. We utilize the same set

20

30

40

50

̄

10 1

EI

PI

MES

x

t

∧

x

0

t

x

t

∨

x

0

t

x

t

▵

x

0

t

x

t

▿

x

0

t

0

50

100

Iteration

10 -2

10 -1

10 0

10 1

Regret

0

50

100

Iteration

10 -2

10 -1

10 0

10 1

Regret

0

50

100

Iteration

10 -1

10 0

10 1

10 2

10 3

Regret

0

10

20

30

40

Iteration

10 0

10 1

Regret

(b) NO3

(c) Phosphorus

(d) Humidity

̄

̄

̄

̄

̄

̄

̄

10 0 of synthetic functions and real-world environmental datasets in the previous section to compare the performance of MP with Var and Rand. The plot of the average and standard error of the maximum regret max k ∈{ 1 , 10 , 20 } ( r π µt ( S µt ( k ) , S c µt ( k )) ) over 15 repeated experiments are shown in Figs. 3m:a-g. The MP algorithm outperforms the other 2 baselines by converging to lower regret. In some active multiple set ordering experiments (e.g., in Figs. 3m:a, 3m:c), the performance gaps between MP and Var are smaller than those in the previous active set ordering experiments (e.g., Figs. 3s:a, 3s:c). It is because estimating multiple topk sets requires more observations, which makes the performance of MP tend towards that of Var which estimates the entire function.

10 -1

5.3 Bayesian Optimization

Regret

10 -2 When k = 1 , the active set ordering problem reduces to the BO problem and a sampling input of our MP algorithm, i.e., x t = ¯ x t ▽ ¯ x ′ t , is the same as that of GP-UCB [19]. Therefore, this section empirically demonstrates the performance of MP with different sampling inputs ( x t ∈ { ¯ x t ▽ ¯ x ′ t , ¯ x t △ ¯ x ′ t , ¯ x t ∨ ¯ x ′ t , ¯ x t ∧ ¯ x ′ t } ) in solving BO. Our aim is not to show that MP achieves the state-of-the-art performance as a BO solver, but rather to demonstrate that it performs comparably to the well-known GP-UCB algorithm. In addition to comparing with GP-UCB (equivalently, MP with x t = ¯ x t ▽ ¯ x ′ t ), we also compare with 3 classical BO solutions: probability of improvement (PI) [13], expected improvement (EI) [15], and max-value entropy search (MES) [21]. The average and the standard error of the regret r π µt ( S µt (1) , S c µt (1)) over 15 repeated experiments are shown in Fig. 4. We observe that MP performs comparably with the well-known GP-UCB algorithm (labelled as ¯ x t ▽ ¯ x ′ t ). Expectedly, EI and MES outperform GP-UCB (and hence, MP) in some experiments such as in Figs. 4b and 4c.

6 Conclusion

10 -3

0 10 20 30 40 50 Iteration This paper presents a new problem formulation, namely active set ordering , that aims to balance between the expensive estimation of the entire function in ED and that of only the maximizers in BO. We propose the mean prediction (MP) algorithm to address this problem with a theoretical no-regret guarantee. Interestingly, BO can be framed as a special instance of active set ordering, which leads to several new subtle understandings regarding the predicted maximizer and other alternative sampling inputs. Last, the performance of MP is empirically evaluated using various synthetic functions and real-world datasets.

Acknowledgments and Disclosure of Funding

This research/project is supported by the National Research Foundation Singapore and DSO National Laboratories under the AI Singapore Programme (AISG Award No: AISG2-RP-2020-018).

DesCartes: this research is supported by the National Research Foundation, Prime Minister's Office, Singapore under its Campus for Research Excellence and Technological Enterprise (CREATE) programme.

This research was partially supported by the Australian Government through the Australian Research Council's Discovery Projects funding scheme (project DP210102798). The views expressed herein are those of the authors and are not necessarily those of the Australian Government or Australian Research Council.

References

I. Bogunovic, J. Scarlett, A. Krause, and V. Cevher. Truncated variance reduction: A unified approach to Bayesian optimization and level-set estimation. In Proc. NIPS , pages 1507-1515, 2016.

E. Brochu, V. M. Cora, and N. De Freitas. A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv:1012.2599 , 2010.

B. Bryan, R. C. Nichol, C. R. Genovese, J. Schneider, C. J. Miller, and L. Wasserman. Active learning for identifying function threshold boundaries. Proc. NeurIPS , 18, 2005.

K. Chaloner and I. Verdinelli. Bayesian experimental design: A review. Statistical science , pages 273-304, 1995.

S. R. Chowdhury and A. Gopalan. On kernelized multi-armed bandits. In Proc. ICML , pages 844-853, 2017.

P. I. Frazier. A tutorial on Bayesian optimization. arXiv preprint arXiv:1807.02811 , 2018.

Roman Garnett. Bayesian Optimization . Cambridge University Press, 2022.

A. Gotovos, N. Casati, G. Hitz, and A. Krause. Active learning for level set estimation. In Proc. IJCAI , 2013.

J. M. Hernández-Lobato, M. W. Hoffman, and Z. Ghahramani. Predictive entropy search for efficient global optimization of black-box functions. In Proc. NIPS , pages 918-926, 2014.

H. Jiang, J. Li, and M. Qiao. Practical algorithms for best-k identification in multi-armed bandits. arXiv preprint arXiv:1705.06894 , 2017.

S. Kalyanakrishnan and P. Stone. Efficient selection of multiple bandit arms: Theory and practice. In Proc. ICML , volume 10, pages 511-518, 2010.

S. Kalyanakrishnan, A. Tewari, P. Auer, and P. Stone. PAC subset selection in stochastic multi-armed bandits. In Proc. ICML , volume 12, pages 655-662, 2012.

H. J. Kushner. A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. Journal of basic engineering , 86(1):97-106, 1964.

B. Mason, R. Camilleri, S. Mukherjee, K. Jamieson, R. Nowak, and L. Jain. Nearly optimal algorithms for level set estimation. In Proc. AISTATS , 2022.

J. Moˇ ckus. The Bayesian approach to global optimization. In System Modeling and Optimization , volume 38, pages 473-481, 1982.

C. E. Rasmussen and C. K. I. Williams. Gaussian processes for machine learning . MIT Press, 2006.

J. Scarlett, I. Bogunovic, and V. Cevher. Lower bounds on regret for noisy Gaussian process bandit optimization. In Conference on Learning Theory , pages 1723-1742, 2017.

Burr Settles. Active learning literature survey. 2009.

N. Srinivas, A. Krause, S. Kakade, and M. Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proc. ICML , pages 1015-1022, 2010.

S. Surjanovic and D. Bingham. Virtual library of simulation experiments: Test functions and datasets. Retrieved May 21, 2024, from http://www.sfu.ca/~ssurjano .

Z. Wang and S. Jegelka. Max-value entropy search for efficient Bayesian optimization. In Proc. ICML , pages 3627-3635, 2017.

R. Webster and M. A. Oliver. Geostatistics for environmental scientists . John Wiley & Sons, 2007.

A Proof of Lemma 3.1

We will show that ρ ( t ) π (˜ x , ˜ x ′ ) is an upper confidence bound of the regret r π (˜ x , ˜ x ′ ) , i.e.,

The regret r π (˜ x , ˜ x ′ ) is defined as follows

Furthermore, from Lemma 2.2, with probability of at least 1 -δ , for all t ≥ 1 ,

Hence, with probability of at least 1 -δ , for all t ≥ 1 ,

Multiplying both sides with (2 π (˜ x , ˜ x ′ ) -1) , we obtain

Therefore, with probability of at least 1 -δ , for all t ≥ 1 ,

B Proof of Lemma 3.5

We note that ✶ µ t (˜ x ) ≥ µ t (˜ x ′ ) = 1 happens when

Therefore,

C Proof of Lemma 3.6

We will show that ρ ( t ) π µt (˜ x , ˜ x ′ ) ≤ |C t ( x t ) | when x t is taken from the set Q t ≜ { ˜ x ▽ ˜ x ′ , ˜ x △ ˜ x ′ , ˜ x ∨ ˜ x ′ , ˜ x ∧ ˜ x ′ } .

Case 1: x t = ˜ x ▽ ˜ x ′ ≜ argmax x ∈{ ˜ x , ˜ x ′ } u t ( x )

where (13) is because ˜ x ▽ ˜ x ′ ≜ argmax x ∈{ ˜ x , ˜ x ′ } u t ( x ) , (14) is because ˜ x ▽ ˜ x ′ ∈ { ˜ x , ˜ x ′ } , (15) is because u t (˜ x ▽ ˜ x ′ ) -l t (˜ x ▽ ˜ x ′ ) ≥ 0 .

Case 2: x t = ˜ x △ ˜ x ′ ≜ argmin x ∈{ ˜ x , ˜ x ′ } l t ( x )

where (18) is because ˜ x △ ˜ x ′ ≜ argmin x ∈{ ˜ x , ˜ x ′ } l t ( x ) , (19) is because ˜ x △ ˜ x ′ ∈ { ˜ x , ˜ x ′ } , (20) is because u t (˜ x △ ˜ x ′ ) -l t (˜ x △ ˜ x ′ ) ≥ 0 .

Case 3: x t = ˜ x ∨ ˜ x ′ ≜ argmax x ∈{ ˜ x , ˜ x ′ } |C t ( x ) | , i.e.,

where (23) is because ˜ x △ ˜ x ′ ∈ { ˜ x , ˜ x ′ } , (24) is from the above proof of case 2 in (21).

Case 4: x t = ˜ x ∧ ˜ x ′ ≜ argmin x ∈{ ˜ x ▽ ˜ x ′ , ˜ x △ ˜ x ′ } |C t ( x ) | . From (16) and (21), it follows that |C t (˜ x ∧ ˜ x ′ ) | ≥ ρ ( t ) π µt (˜ x , ˜ x ′ ) .

D Proof of Theorem 3.7

By choosing x t following Lemma 3.6, with probability of at least 1 -δ , for all t ≥ 1 ,

Hence, with probability of at least 1 -δ , for all T ≥ 1 ,

Since β t is a non-decreasing sequence, β t ≤ β T for all t ≤ T . Therefore, with probability of at least 1 -δ , for all T ≥ 1 ,

Hence, with probability of at least 1 -δ , for all T ≥ 1 ,

From Lemma 4 in [5], i.e.,

E Bayesian Optimization as Active Set Ordering with k = 1

E.1 Regret

For a predicted top1 set S µ t (1) , π µ t ( S µ t (1) , S c µ t (1)) = 1 . Hence, the regret for predicting S µ t (1) is expressed as follows.

Let S µ t (1) ≜ { ˆ x ∗ } and x ∗ ∈ argmax x ∈X f ( x ) .

Since X = S µ t (1) ∪ S c µ t (1) , there are 2 cases

If x ∗ ∈ S c µ t (1) , then max x ∈S c µt (1) f ( x ) = f ( x ∗ ) ≥ f (ˆ x ∗ ) and

If x ∗ ∈ S µ t (1) , i.e., x ∗ = ˆ x ∗ , then max x ∈S c µt (1) f ( x ) ≤ f (ˆ x ∗ ) and

When k = 1 , due to the definition of S µ t ( k ) in Definition 4.2

Case 2: If u t (ˆ x t ) = u t (¯ x t ) which implies that ˆ x t ∈ S c µ t ( k ) and u t (ˆ x t ) > u t (¯ x t ) , then we prove that u t (ˆ x t ) = u t (¯ x ′ t ) by contradiction. Assuming that

Let us consider the following upper confidence bounds of pairwise orderings:

E.2 Sampling Input

We will show that

Let

We consider the following 2 cases:

Case 1: If u t (ˆ x t ) = u t (¯ x t ) , then

̸

Furthermore, from the choice of (¯ x t , ¯ x ′ t ) in (11) and ˆ x t ∈ S c µ t ( k ) ,

Hence,

which contradicts to the assumption 25. Therefore, u t (ˆ x t ) = u t (¯ x ′ t ) and

F Upper Confidence Bound of the Regret r π ( X 0 , X 1 )

With probability of at least 1 -δ , for all t ≥ 1 and for all { x , x ′ } ⊂ X , r π ( x , x ′ )= π ( X 0 , X 1 ) ≤ ρ ( t ) π ( x , x ′ )= π ( X 0 , X 1 ) . Therefore, with probability of at least 1 -δ , for all t ≥ 1 and for all { x , x ′ } ⊂ X ,

G Transitivity of Pairwise Orderings

The transitivity property of the binary relation µ π t follows directly from its connection to the GP posterior mean in Lemma 3.5. In particular, we would like to show that if

From Lemma 3.5, the premise (26) implies that

Applying Lemma 3.5 again, we conclude

then which implies that

H Proof of Lemma 4.3

Applying Lemma 3.6 to the input pair (¯ x t , ¯ x ′ t ) , by selecting x t ∈ { ¯ x t ▽ ¯ x ′ t , ¯ x t △ ¯ x ′ t , ¯ x t ∨ ¯ x ′ t , ¯ x t ∧ ¯ x ′ t } ,

Furthermore, from the choice of (¯ x t , ¯ x ′ t ) in (11),

Therefore,

Algorithm 2 Mean Prediction (MP) for Active Multiple Set Ordering

Require: X , D 0 , { k 1 , k 2 , . . . , k m } , T

1: for t = 1 to T do

2: Update GP posterior belief: { µ t ( x ) } x ∈X , { σ t ( x ) } x ∈X .

3: Construct {S µ t ( k i ) } i m =1 as the collection of m topk sets predicted using µ t . ▷ Prediction

4: ( ¯ ¯ x t , ¯ ¯ x ′ t ) ≜ argmax ( x , x ′ ) ∈∪ i m =1 S µt ( k i ) ×S c µt ( k i ) ρ ( t ) π µt ( x , x ′ )

5: Select x t ∈ { ¯ x t ▽ ¯ x t , ¯ x t △ ¯ x t , ¯ x t ∨ ¯ x t , ¯ x t ∧ ¯ x t } .

¯ ¯ ′ ¯ ¯ ′ ¯ ¯ ′ ¯ ¯ ′ ▷ Sampling input

6: y t ( D t ) ← y ( D t -1 ) ∪ { y ( x t ) }

7: end for

8: Update GP posterior belief: { µ T +1 ( x ) } x ∈X , { σ T +1 ( x ) } x ∈X .

10: return {S µ T +1 ( k i ) } i m =1 .

9: Construct {S µ T +1 ( k i ) } i m =1 as the collection of m topk sets predicted using µ T +1 .

I Proof of Theorem 4.4

By choosing x t following Lemma 4.3, with probability of at least 1 -δ , for all t ≥ 1 ,

Furthermore, from the non-decreasing property of the sequence ( β t ) T t =1 ,

By utilizing the result from [5] like Appendix D, we can obtain

Therefore, with probability of at least 1 -δ , for all T ≥ 1 ,

J Active Multiple Set Ordering

The pseudocode of MP algorithm for the active multiple set ordering problem is shown in Algorithm 2. In the rest of this section, we prove the cumulative regret bound of Algorithm 2.

We recall that in (12),

Therefore,

Furthermore, applying Lemma 3.6 to the input pair ( ¯ ¯ x t , ¯ ¯ x ′ t ) , for any x t ∈ { ¯ ¯ x t ▽ ¯ ¯ x ′ t , ¯ ¯ x t △ ¯ ¯ x ′ t , ¯ ¯ x t ∨ ¯ ¯ x ′ t , ¯ ¯ x t ∧ ¯ ¯ x ′ t }

i.e.,

Therefore,

Hence, with probability of at least 1 -δ , for all t ≥ 1 ,

As a result, with probability of at least 1 -δ , for all T ≥ 1 ,

where the last inequality comes from Appendix I.

K Experiments

All experiments were conducted on a computer equipped with an AMD Ryzen 7 6800HS processor and 16GB of RAM.

To generate a function sampled from a GP, we randomly generate 3 observations { ( x i , y ( x i )) } 3 i =1 and fit a GP model to these observations. Then, we sample function evaluations at all inputs in the domain from the GP posterior distribution. These function evaluations are considered the evaluations of the blackbox function. To generate an observation at a sampling input, we add a Gaussian noise (of σ n = 0 . 1 ) to the evaluations of the blackbox function at the sampling input.

The expressions for the Branin-Hoo and Goldstein-Price functions are described at [20]. We transform the input domain of these functions to [0 , 1] 2 and standardize the function evaluations. The input domain is discretized into n = 100 points randomly selected in the domain [0 , 1] 2 . The noise is chosen with σ n = 0 . 1 .

To perform experiments with the NO3 dataset from Lake Zurich (available at https://wldb.ilec. or.jp/Lake/EUR-06/datalist ), we standardize the NO3 measurements. Then, a GP model is trained on the standardized dataset to generate the noisy evaluations of the blackbox function over n = 100 randomly chosen locations.

We use the logarithmic values of the phosphorus measurements in the soy survey of Brooms Barn [22] to construct a blackbox function. The locations are normalized to the range [0 , 1] 2 and the logarithmic values of the phosphorus measurements are standardized. Then, we train a GP model to generate the noisy evaluations of the blackbox function over n = 400 randomly chosen locations.

To perform experiments with the humidity dataset, we extract the humidity measurements at different locations with the same mote id of 31167 from the Intel Lab data (available at https://db.csail. mit.edu/labdata/labdata.html ). The humidity measurements are standardized. Then, a GP model is trained to this extracted dataset to generate the noisy evaluations of the blackbox function over n = 100 randomly chosen locations.

To remove the potential inefficiency due to repeated sampling in Rand and Var, we provide additional experiments by replacing the Rand and Var baselines with RandNoRepl and VarNoRepl, which do not allow repeated sampling. This modification potentially gives RandNoRepl (random sampling without replacement across different iterations) and VarNoRepl (uncertainty sampling without replacement across different iterations) an additional advantage over our solutions which allow repeated sampling. By avoiding repeated sampling, RandNoRepl and VarNoRepl can sample the input domain more uniformly, whereas our methods might re-sample certain input regions. However, as shown in Figure 5, RandNoRepl and VarNoRepl still do not outperform our solutions. The justification for the efficiency of our solutions is in the nature of noisy observations: With a noise standard deviation of σ n = 0 . 1 , a single observation at each input may not suffice to accurately determine the ordering with its neighboring inputs in terms of the function value. Hence, spreading the sampling budget across the whole input domain may not perform well. In contrast, our approach allocates more sampling inputs to the boundary of the topk set, where it is particularly challenging to check if inputs belong to the topk set (see Figure 2).

Regret

10 2

10 1

10 0

10 -1

10 -2

̄

Rand

Var

VarNoRepl

x

t

∧

x

0

t

x

t

∨

x

0

t

x

t

▵

x

0

t

x

t

▿

x

0

t

0

50

100

Iteration

10 -1

10 0

10 1

10 2

Regret

0

50

100

Iteration

10 -2

10 -1

10 0

10 1

10 2

Regret

(b) Goldstein-Price

̄

̄

̄

̄

̄

̄

̄

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Our contributions include introducing the active set ordering problem, designing a novel solution with theoretical performance guarantee, and investigating a connection to Bayesian optimization. The experiments are performed on environmental monitoring datasets which aligns with the motivation in the introduction.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

20 40 60 80 100 Iteration · It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: The limitations are elaborated as assumptions in the paper including: a finite input domain of the blackbox function (see Sec. 2.1) and the blackbox function belonging to a reproducing kernel Hilbert space with a bounded norm (see Sec. 2.2).

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

0

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory Assumptions and Proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: We have provided the proofs for all theoretical results in the appendix of the paper.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental Result Reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification:We have described the experiments and provided the code and datasets for reproducing the experimental results.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same

dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We have provided the source code, datasets, and scripts to reproduce the experiment results.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental Setting/Details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We have provided the URLs where datasets are downloaded and described parameters used in the experiments such as the size of the input domain and the noise variance. Other parameters such as the random seed are configured in the submitted code.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment Statistical Significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We have repeated the experiments with different random seeds and reported the average and standard error of the experiment results.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments Compute Resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We describe the computer resources in Appendix K.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code Of Ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We ensure that the research conforms with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader Impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: Our work proposes a general active learning problem that does not have any direct path to any negative applications.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The paper poses no such risks.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We have cited the sources and/or provided the URLs to all public datasets used in the paper.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New Assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: The paper does not release new assets.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and Research with Human Subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The paper does not involve crowdsourcing nor research with human subjects.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis submission proposes a novel Mean Prediction (MP) method for active set ordering problem. MP selects pre-defined k inputs with highest Gaussian Process posterior mean values through a novel sampling strategy. Theoretical analysis on the regret, the prediction and the sampling strategy of proposed method is discussed in details, and experiments on synthetic functions and real-life application prove the effectiveness of MP for ordering sets of inputs based on expensive evaluation on black-box function. The submission includes GP-UCB Bayesian optimization as a special case of active set ordering problem.\n\nSTRENGTHS:\n1.  The paper presents adequate theoretical justification for their proposed method. The motivation for studying the active set ordering problem is clearly stated and the proofs on regret bound, prediction and sampling strategy seem valid to me.\n\n2.  The idea of formulating black-box function optimization into set ordering problem presents novelty to some extend. It provides some alternative sampling strategy by proposing the input set $Q_t$ in lines 145-146, which can be more informative than traditional metric such as sampling based on maximizing GP variance.\n\nWEAKNESSES:\n1.  My first concern regards the dimensionality of the problem setting. The theoretical justification of MP seems valid for any value of $d$ where $d$ is the dimension of the black-box function. However, in the Experiments section all the problems are 2-dimensional, which is insufficient to show how practical the proposed MP method could be in applications. \n\n2.  This sampling-based method only works on discrete space problems to me. The authors use classic GP model on continuous space as the surrogate model to compute posterior mean, but then sample on the discretized space during inference. This could be one of the major limitations of MP method because a lot of probabilistic information has been missed during the discretization.\n\nQUESTIONS:\n1.  I'm not very convinced by the experiment design. For example the experiment on Goldstein-Price function (lines 248-251), the input domain is discretized into 100 points and a top-5 set based on posterior mean is sampled at each of the 100 iterations. Even repeated sampling is allowed in this paper, I still feel like almost the whole space which only consists 100 points has been sampled by doing so. How do the authors justify the efficiency compared to brute force sampling over the whole sample space?\n\n2.  In all experiment settings, the 100/400 points that represent the input domain $\\mathbf{X}$ are randomly sampled from normalized domain $[0, 1]^2$. If I understand correctly, each $\\mathbf{X}$ of the 15 repeated experiments is different, which lead to 15 different $S_{\\mu_t}(5)$ at the end of iterations. How do the authors decide which one of the $S_{\\mu_t}(5)$ is the true top-5 set for the objective black-box problem? (e.g. which 5 locations are the top-5 with highest $NO_3$ concentration in Lake Zurich?)\n\n3.  Could the authors explain how $S_{\\mu_t}(k)$ is constructed in line 3 of Algorithm 1? Are the posterior mean of all the points in $\\mathbf{X}$ evaluated? Similar for $(\\mathbf{\\bar{x}_t}, \\mathbf{\\bar{x}'_t})$ (line 4), are all possible pairs in $S_{\\mu_t}(k)$ evaluated?\n\nLIMITATIONS:\nSee above for limitations.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2024-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper poses a novel active learning problem formulation of active set ordering, in which we aim to identify the data points that yield the top- and bottom-$k$ values via a given objective function.\nThis active learning goal serves as a compromise between Bayesian experimental design which focuses solely on learning and Bayesian optimization targeting optimization, and poses as an alternative to level-set estimation, especially in scenarios where a level-set threshold is not easily determined.\nThe authors first define an appropriate metric of regret, propose using the mean prediction of the surrogate Gaussian process to generate an ordering of the data points to recommend to the user, and finally develop an acquisition function that aims to reduce an approximation of the regret resulting from that posterior predictive ordering.\nThe paper presents various theoretical results bounding the regret of the proposed algorithm, and experiments are conducted to illustrate the empirical effectiveness of the algorithm.\n\nSTRENGTHS:\nI find active set ordering to be an interesting active learning problem that is related to other common active learning problems (experimental design, level-set estimation, Bayesian optimization).\nThe various algorithmic choices made throughout the paper are reasonable and well-motivated by theoretical insights.\nThe experiments include a wide range of synthetic and real-world tasks, and do a good job showing that the proposed strategy yields competitive performance against baselines.\n\nWEAKNESSES:\n- The presentation of Sections 3 and 4 is a bit hard to follow.\nI understand Section 3 serves as the base case where we develop the core tools which are then further extended in Section 4, but the narrative has a jumping-back-and-forth feel that I find somewhat confusing.\n- The claims the authors make in Section 1 to motivate the problem might be too strong.\nI would say something along the lines of, this work presents an alternative to LSE when setting the threshold is difficult (as opposed to that it should replace it, since sometimes a desirable threshold is known, in which case LSE should be preferred).\n- I would include LSE acquisition functions (e.g., STRADDLE) among the baselines in the experiments.\n\nQUESTIONS:\n- The search space sizes in the experiments are quite small.\nDoes the algorithm scale well to large search spaces?\nWhat’s the computational complexity?\nDoes it take quadratic time with respect to search space size (because we need to iterate over all pairs)?\n- Any insights on how the different variants behave and which should be preferred in which situations?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-11"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper generalizes the best $k$-arm identification problem to Gaussian processes with the goal to estimate the set of the best $k$ function evaluations $f(x)$ on a finite domain $X$, where $k=1$ corresponds to standard Bayesian optimization. The proposed regret notion is a natural adaptation of the common regret $f(x*)-f(x_t)$ in Gaussian process bandits.\n\nThe paper is accompanied by some first experiments on benchmark datasets.\n\nSTRENGTHS:\nInteresting problem.\n\n---- rebuttal ----\nchanged from 5 to 6.\n\nWEAKNESSES:\nFiniteness assumption seems rather strong and limits this work. Also, the significance of the results (Thm 3.7 and 4.4) is not fully clear to me, as the paper seems to be mostly relying on known techniques (for kernelized bandits, and Gaussian process bandits), while the related work is not sufficiently well discussed (see questions and limitations).\n\nAm happy to raise my score, if my concerns are addressed. Specifically about the comparison to previous and related literature.\n\nThe definition of $\\pi_*(X_0,X_1)$ seems only partial, i.e., what if there exists some $x_0$ that are larger than some $x_1$ and some $x_0'$ that are smaller than some $x_1'$\n\nMinor comments:\n* The notation is a bit convoluted. E.g., do you really need double bars and dashes for $x$. Even just tilde&dash is not great. E.g., in the beginning of section 3 you could just do $x,x'$ without the tilde (or you use $y$ etc.).\n* Please do not mix definitions (in particular of important notation) and lemmas (e.g., Lemma 3.6, 3.1, ...).\n\nQUESTIONS:\nWhat is the computational complexity/runtime of your approach? Is the runtime exponential in $k$?\n\nHow does your work relate to Gaussian bandit papers with correlated/dependent arms (e.g., Gupta et al. 2021, Pandrey et al 2007)\n\nHow does the results here relate to e.g., the \"Interactive submodular bandit\" by Chen et al. [ICML, 2017] (and related), where a submodular function is maximized in a similar fashion (in a GP-style context). Can your $k$ best selection problem be cast as a submodular (or even simply modular) set optimization problem? Are the two regret variants comparable?\n\nLIMITATIONS:\nThe finiteness assumption on $X$ seems rather restrictive. Can $X$ be countable, or even say a compact subset / interval? Standard work on Gaussian processes with regret bounds (Srinivas et al., etc.) does not have this restrictions. For finite $X$ less assumptions might be possible, see e.g., Theorem 1 by Krause & Ong (NeurIPS 2011) (for example, no explicit assumptions on the rkhs norm as you do in Lemma 2.2, which they only require for arbitrary domains $X$).\n\nAlso in general the related work is not discussed sufficiently well, see questions.\n\nPlease, also see the questions.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 2, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-12"}

comments: SUMMARY:
This paper introduces the "active set ordering" problem, which aims at recovering the top-k actions in a set by strategically sampling actions. The authors formally define the problem in the regret minimization setting and propose an algorithm for that. They authors upper bound the regret and run experiments the test the proposed algorithm.

STRENGTHS:
The authors introduced the problem of recovering top-k actions in the regret minimization setting, and developed an algorithm for that. The authors theoretically show their algorithm enjoys a \sqrt{T} type of regret (after ignoring some problem dependent quantities). Experimental results show the proposed algorithm has good performance.

WEAKNESSES:
1. The authors didn't explicitly quantify some problem-dependent quantities in their regret bound, e.g., \gamma_T and \beta_T. How large these quantities are in different settings? 
2. A lower bound analysis is missing, which make it even harder to know if the upper bound is tight or not.
3. While I understand the studied setting is slightly different from top-k arm identification, the proposed algorithm is actually quite similar to algorithms proposed in top-k arm identification: I believe some appropriate acknowledgement is missing in the paper.
4. The wording "set ordering" in misleading: the goal of this paper is not to recover the set ordering but simply identifying the top-k actions in the regret minimization setting.

QUESTIONS:
See comments above.

LIMITATIONS:
N/A
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-13