Practical Bayesian Optimization of Objectives with Conditioning Variables
Authors: Michael Arthur Leopold Pearce
Janis Klaise
Matthew Groves
Venue: NeurIPS 2021 Submitted (2021)
Keywords: Gaussian processes
optimization

Abstract: Bayesian optimization is a class of data efficient model based algorithms typically
focused on global optimization. We consider the more general case where a user is
faced with multiple problems that each need to be optimized conditional on a state
variable, for example given a range of cities with different patient distributions,
we optimize the ambulance locations conditioned on patient distribution. Given partitions of Cifar10,
we optimize CNN hyperparameters for each partition. Similarity
across objectives boosts optimization of each objective in two ways: in modelling by
data sharing across objectives, and also in acquisition by quantifying how a single
point on one objective can provide benefit to all objectives. For this we propose a framework
for conditional optimization: ConBO. This can be built on top of a range of acquisition functions
and we propose a new Hybrid Knowledge Gradient acquisition function. The resulting method is intuitive and
theoretically grounded, performs either similar to or significantly better than recently published works
on a range of problems, and is easily parallelized to collect a batch of points.

Practical Bayesian Optimization of Objectives with Conditioning Variables
Anonymous Author(s)
Affiliation Address email
Abstract
Bayesian optimization is class of data efficient model based algorithms typically focused on global optimization. We consider the more general case where a user is faced with multiple tasks that each need to be optimized, find the global optima within each task, all the task conditional optima. For example given a range cities with different patient distributions, each and every city; given subclass partitions of CIFAR-10, we optimize CNN hyperparameters for each partition. Similarity across tasks boosts optimization of each task in two ways: in modelling by data sharing across objectives, and also acquisition by quantifying how single point one task can help Iearn the optima similar tasks. For this we propose framework for conditional optimization: ConBO. This can be built top of range of acquisition functions and propose a new Hybrid Knowledge Gradient acquisition function. The resulting method is intuitive and theoretically grounded, either matches significantly outperforms recently published works on a range of problems, and thanks to the unique nature of conditional optimization, is easily parallelized to collect batch of points.
2
optimize the ambulance locations for
6
8
10
12
14
15
Introduction
17 Expensive stochastic black box functions arise in many fields such as fluid simulations [ engineering wing design (2), and machine learning parameter tuning [3]. Bayesian optimization is powerful set 19 of tools to optimize such functions, finding the input with highest long term average performance x² arg max, (x:), (the expectation represents averaging over the performance noise). In this
21 work we consider an under-explored generalization of the standard setting previously referred to as "conditional optimization" 4 where user has collection of functions, tasks, and simply seeks 23 the peak of each function/task. Formally, we have an expensive black box f that takes as arguments
24 both task s S and imput x E X, typically box-constrained continuous variables, and returns noisy scalar performance
25
f(s,z):
26 At each iteration algorithm determines both task and input (s,2r) then observes performance f(8 and the goal is to learn the input with highest average performance for each task
27
x (s) arg max (s,
28 (8) referred to as the optima conditioned on 8, see Figure black line. In certain applications,
29 one may want to give higher priority to particular tasks hence task weighting function W(s) may
30 also be specified. In this work we consider the following applications.
31 CNN hyperparameters: the CIFAR-10 10 dataset contains 10 classes, this is split into five mutually
32
exclusive binary classification datasets and for each we train a CNN, each CNN is task in S
Submitted to 35th Conference on Neural Information Processing Systems (NeurIPS 2021). Do not distribute.

---

46 cities, like Singapore or Dubai, with a popula- tion centre a boundary. Thus, (s) mirrors 33 {1...., 5}. For each CNN, we optimize dropout rates, batch size and Adam parameters, so X C R
34 and f(s, x) is the validation accuracy. We assume all five CNNs have equal priority, W W(s) = 1/5.
35
Ambulances in a square: [5] given a range of 30kmx30km cities, each city is a task with
36
Negative Rosenbrock Function
37
different population centre s E [0.30] 30 For
3.0
38 a given city, we optimize the 2D location of three ambulance bases, X E [0, 30 C R Given a city s and ambulance bases H, a vir- tual environment randomly generates patients for a simulated day and the average ambulance journey time, f(s, z), is returned. Inland cities, 44 like Paris or London, with population centres
39
2.0
0
40
1.5-
41
Action
42
0.5
43
0.0
0
data
max (5.
45 in the middle are more common than coastal
-0.5
(s,x)
tasks
-1.0
47
State Value
4-
48 this distribution of city centres, a Gaussian cen- tred at (15, 15) with 7km standard deviation and truncated to [0, 30]2.
2-
50
2.0
-1.5
-1.0
-0.5
0.0 Task
0.5
1.0
1.5
2.0
51
Assemble to order: 66 7 a company owns
many stock warehouses and each one faces different level of demand E [0.5, 1.5]. Equal priority is given to all warehouses and so W(s) 55 is uniform. At each warehouse, stock levels are controlled by setting targets of a control pol- icy = [0, 20 Given demand level and 58 control parameters x, a simulator generates cus- tomer orders according to demand, the ware- house stock is sold and replenished according to policy x, and profit over a simulated month is returned as fs.
52
Figure 1: Top: GP model. Each task (vertical slice) is an objective function to maximise. new sample (5)x))1 provides information about the optima of similar tasks. Bottom: the acquisition function value of each task using hybrid Knowl- edge Gradient.
53
54
56
60
61
62
The closely related multi-task Bayesian optimisation or Bayesian Quadrature optimisation methods 10 11] aim to find a single peak that maximises the weighted sum/integral over tasks, x² arg max, v [[f(ss =)))(sss However, as they are fundamentally designed for a different purpose, we observe that they struggle to find the peak of each and every task, see experiments in Supplementary
63
8, 9
64
66
Material 6 (SM 6).
87
conditional optimization has also been referred as "offline contextual" [15]. In contextual applications, at iteration the next task (called context) s" is passed from the black box to a contextual algorithm that intelligently determines z". Then the black box returns both performance y" and the next task/context + One could "hotfix" contextual algorithm for conditional optimization problems by randomly choosing sn+1 ~ [Ps)] x W(s) at each iteration. However, we empirically show that, unsurprisingly. randomly choosing the task in each iteration is significantly worse than intelligently choosing the task.
Also closely related are contextual optimization algorithms 12
68
89
70
71
72
73
74
Multi-fidelity and multi-information source methods [66 17 18 [9] assume there is a single known constant target task 8 E S corresponding to the highest fidelity level or most expensive information source that must be optimized, x¹ arg max, [[ x)]. If the cost of evaluation varies across the function domain, c(s, r), cheaper regions of the domain can be evaluated improving sample efficiency. Again, such methods may be hotfixed for the setting we consider by artificially designating a target task, c.g. the highest priority task s* arg max, W (s). However, we observe that if cost is constant across the domain (as is commonly assumed in the BO literature), these methods greedily optimise the artificial target task s* and blindly neglect all other tasks, see SM4. This desirable behaviour in multi fidelity optimization problems leads to failure in conditional optimization problems.
75
76
77
78
79
80
81
82
83
To the best of our knowledge, there exist only small number of works that propose algorithms specifically designed for the conditional setting.
84
The Surrogate Collaborative Tuning (SCoT) algorithm (20 optimizes a finite set of tasks. It iteratively 86 visits each task, s, in a round-robin fashion and determines the input x by expected improvement (EI). The authors apply this to optimize the hyperparameters of an ML model for multiple datasets. The
85
87
2

---

88
Profile Expected Improvement (PEI) and Profile Expected Quantile Improvment (PEQI) algorithms
89 21 4 consider continuous tasks and significantly improve upon SCoT by dynamically determining
90
the task in each iteration. The acquisition value of given point (s,ar) is the expected improvement of a new output over the best predicted output within the same task.
91
92
The REVI algorithm [221 at each iteration discretizes both task S' and input X spaces. Given a new hypothetical point, 25, for each task in the discretization, 84, the discrete Knowledge Gradient over the X discretization quantifies how (8, 7) n*+1 will benefit task 8;. Summing the benefits over the tasks yields the acquisition value of (s. 2)) -=====+++____= REVI was designed account for how all tasks can be optimized by each sample. However, this comes with exponential cost, increasing problem dimension leads to exponentially increasing discretization size with corresponding exponential space and time requirements. On the other hand, sparse discretizations can lead to poor and arbitrary measures of acquisition value. Consequently, REVT has only successfully been applied to low-dimensional synthetic problems.
93
94
95
96
97
98
99
100
101
Most recently, the Multi-task Thompson sampling (MTS) method [15] uses novel kernel with length scale that varies across tasks. This is combined with a Thompson sampling method for col- lecting new data and showed performance improvements over REVI. However, Thompson sampling also does not account for how one sample provides benefit for optimizing all tasks, fundamental structural property of conditional optimization.
102
103
104
105
106
We propose a method that exploits the basic structure of conditional optimization while also being highly scalable, and therefore applicable to more challenging real-world problems. We make the following contributions:
107
108
109
1. A framework for conditional Bayesian optimization with theoretical guarantees: ConBO.
110
2. A new, fast global optimization method: Hybrid Knowledge Gradient.
111
3. State-of-the-art performance on open source problems including CNNs and simulators.
The Conditional Bayesian Optimization Algorithm
112
115
We first discuss the fitting of the Gaussian process model and the predicted conditional optima. We then motivate the acquisition value for a single task and how this is integrated over tasks yielding the acquisition function. Because integrating over tasks multiplies the computational burden, we propose Hybrid Knowledge Gradient as a solution. See Algorithm in the SM 2 for a high level summary.
114
116
At stage after having observed n data points, {(s", x², (²) where s² = f(s",=")) Gaussian ( ...., (s, x)") and
117
process is fit over the joint space S x X to scalar outputs y. Let X
118
119 Y = (y) ".......,, A Gaussian process is defined by a prior mean and prior covariance function, (s, (s), (s',z')) 2²)) which are chosen for each application. for more information see (23).
Let the data covariance matrix be K- R0 X²) E R the posterior mean is given by
(² = 1a (s, + h ((s, x). (²) +) - 1 ,x((²))
122 and the posterior covariance is given by
km ((,, ('),,')')))=))>)))))))")))))(( < x), (8', x²)) + k"(((s Xx), ( +o/) 10("",
-x (5',
123 At the end of data collection, in standard BO, often the peak posterior mean is returned as the
124
predicted best input. Generalizing to the conditional setting we simply condition on s,
(8) = arg max p N (8, 2)
x
125
During data collection, a new data point + at (s, -) will update the Gaussian process over the whole domain S x X. To construct an acquisition function for conditional optimization, we start by looking for standard acquisition functions that account for how the model changes unsampled locations (s', x') # (s, (x)++++++ Specifically, the popular Expected improvement (EI) (44 and upper confidence bound (UCB) (25) methods are both functions of the mean and kernel at the sampled point
126
127
128
129
only, i.e. of (1"((s, .r) =r) k- ((x, x²) n+11 (x, (x. r))++)), hence would require non-trivial modification
131
to be able to account for how a sample affects similar tasks. Methods that do utilise the mean and
132
kernel at unsampled points include Entropy search (ES (26. 27 and PES (8) that measures the
3

---

Foster Moan Updates
KG by diccretziing X
KG by over
Hybrid KG
2-
samples
*oo BLaX ph+
2-
of *po
maa, ol 1p0
0000000
0-
6
0-
-2
-2
Figure 2: Methods for computing (G(x*+1) at x=+1 = 7. Left: (x) and samples of n² an+1( (2)
determined by a scalar Z ~ (O, Centre-left: KGa replaces X with up to 3000 points 24 E Xa and ( is linear in Z. Centre-right: KGwC samples up to 1000 functions y=+1(r) functions and maximises each of them numerically. Right: KG, samples up to 5 functions 1zm (a) and maximizes them numerically, the arg max points .5, are used as Xa in KG..
mutual information between the new output P[y [x=+]] and the (uncertain) location of the peak P(z" [X"', y"], Max-value entropy search (MES) 22 that measures mutual information between
133
134
the new output Ply Fly*+*/2*** and the (uncertain) largest output Pfmax y| X Y*], and Knowledge Gradient (KG) [30] that measures the expected peak of the new posterior mean E[max u"+ (x)]
135
caused by a new gu+ at 1+1
137
138
Each single task s defines a single global optimization problem over x E X. Given proposed
sample (s, =+) (s*+1, ,x*+1), ), the acquisition benefit for task s may be computed using ES, PES,
139
140
MES or KG. In this work we adopt KG for its Bayesian decision theoretic derivation that extends
141
seamlessly to the conditional setting. For KG, the benefit for given task is the expected increase in
peak predicted performance within the task. We denote the task-conditioned KG as
(se (s,z)*+1)= = +s [max u (s, x1,, max μz (s,
143
We discuss numerical evaluation of (G,()) in Section 2.1
Similar expressions for conditioned entropy methods, E-S(.). PES,(.), MES,(). are derived in SM 7. Integrating over all tasks yields the total acquisition value
144
145
/ (G, (1, (s)ds.
(4)
For discrete S the integral is replaced by summation. For continuous S, the integral over tasks cannot be computed analytically so we use Monte Carlo with importance sampling. When using kernel that factorises k(s, x, 8²,2²) ( s kx) (z,2²),, like squared exponential Matérn, similarity across tasks is encoded in ks(s, s'). This naturally leads to the proposal distribution (() x (s,s, sn* In our continuous task experiments, we use the Matérn kernel and Gaussian proposal distribution with mean + and the task kernel length scales, l,, as standard
147
149
151
deviations,
a(s|oat2) ~ (N(s6**t1 diag((")).
(5)
153
We generate n, 20 samples SMC = {81, 3n finally the acquisition function is
W(s))
ConBO(s -=+1)=
KG (s,x))+1).
((s:|s*+])
S:ESMC
shows set of sampled tasks and the KG,(-) for each one. Each KG term directly measures increase predicted performance for one task, e.g. if y values are dollar amounts, ConBO with K..(.) is the sum of dollar increases over all tasks. However, for entropy based methods, ConBO becomes a sum of Shannon information units thereby indirectly optimizing the dollar amounts. The randomly sampled tasks SMC may be resampled with each call to ConBO(s, xx) and gradients estimated enabling the optimal (s, xx) a-+ to be found with a stochastic gradient ascent optimizer such as Adam [3I] Selecting each point according to maximising ConBO is also myopically optimal in value of information framework:
Figure
155
157
159
161
Theorem Let "*,x*) arg max ConBO
s,,) x.) be point chosen for sampling. (s*,z*) is also the point that maximises the myopic Value of Information, the increase in predicted performance.
163
Further, in finite search space, with an infinite sampling budget all points will be sampled infinitely:

---

Algorithm Computing ConBO(s, v:) For each call with candidate (task, input) point, similar tasks are sampled. For each sampled task a cheap acquisition function, hybrid KG, evaluated. The output is the importance weighted average of hybrid KG values. Details and full numerical expressions are given in SM 3.
Require: candidate point (s,x) 1 parameters Ns. n2 Sample nt, tasks similar to s SMC ~ ((ssst))) Initialize output value Q 0-0
parfor in SMC do
Initialize X 0
Hybrid Knowledge Gradient
parfor j in 1. n2 do
2j $ 3--10 (22 ((2j-1)/2m)) 1)/2n=)
at1 computed with (s, x))=1 and 2j
yj
} +1 (s) constructed by rank-1 update with (s, (8) 1+)
43 - arg max (s,, with Optimizer(
X X; Us;
end parfor
a: - (X,)
Q - Q + as (ss)) / (114)/(()//0*1))) (11 end parfor return average over tasks Q/n²
Theorem 2 Let S and X be finite sets and N the budget to be sequentially allocated by ConBO. 186 Let n(s, x, N) be the number of samples allocated to a point (s.x) within budget N. Then for all x) E S X we have that lim x- n(s,2; 2, N) 00.
167
The law of large numbers ensures that the algorithm learns the true expected performance for all points. Proofs are given in the SM
189
170
2.1
Hybrid Knowledge Gradient
By definition, KG is more expensive than El and UCB. Further, the function KGr(*i, (s) x))++))) must be computed once for each sampled task 8;, the computational cost is therefore n, times the global acquisition function equivalent. To alleviate this cost, we propose a novel, efficient algorithm for computing KG. In the following section we assume constant s for brevity, reducing to the global optimization setting, Given a hypothetical location -=+1, KG quantifies the value of new hypothetical observation y n=+1 by the expected increase in the peak of the posterior mean
171
179
174
175
178
KG(r*+1)+ = Eyni max 1t" 7+1 (x') x+
max Ju" (x"). z=f'
(6)
177 However, max,' (1 (x') has no explicit formula and approximations are required which we describe
next. At time n, the next posterior mean is unknown, however, it may be written as ==* (x:
179 1=" (x) + (z;; (x;x x*+) where ö ((x; 1= ) deterministic function and the scalar Z ~ (0, 1)
180
captures the randomness of y "+1 see SM 2. Previously, KG(x) has been computed in ways.
KG by discretization 30 32] in Equation 6 the maximizations may be performed over a discrete
set of d points Xa C X. Denoting μ nr (Xa) E Rd and ă ((z*+1) o(X,;r*+1) Rd then
KGa(#*+1) Ez [max ((x*+11 }3}
max Jr.
The max + ((*++))Z) is a piece-wise linear function of Z and the expectation is analytically tractable. The output is lower bound of the true KG(z). REVI [22) and the MiSo algorithm 177 used KG,, with 3000 uniformly random distributed points. This method suffers the curse of dimensionality X, must grow exponentially with dimension. Further, the discretization may likely contain many useless points in uneventful regions of X, see Figure eegureeeeentrrn centre-left plot.
183
184
185
5

---

KG
Type of Estimate
n2 3
n2 =5 5
72 =
n2 50
Discrete MC
lower bound
0.24 (1.37)
0.41 (1.49)
0.55 (1.91)
2.03 (2.2)
unbiased
3.54 (4.79)
3.73 (4.51)
3,50 (3.39)
3.36 (1.18)
MC++
unbiased
2.97 (3.78)
3,50 (2.68)
3.22 (1.69)
3.32 (0.2)
Hybrid
lower bound
3.15 (0.00)
3.28 (0.00)
3.31 (0.00)
3.34 (0.00)
Table 1: We collect 20 points on the Rosenbrock function, point was randomly selected and KG computed by the different methods. Each calculation is repeated 50 times and we report mean and two standard deviations. The Monte Carlo methods suffer from high variance, the discrete method is volatile and returns a very loose lower bound. Hybrid KG is very stable with errors too small to show.
187
given x=+11, the method samples up to n2 = 1000 Gaussian values of Z. For each Zy, construct the posterior mean, n² (z), find the maximum with a continuous
KG by Monte Carlo 33 34
188
189
numerical Optimizer () like L-BFGS CG. Averaging the maxima from all Zj yields
KGAc (a n = - Σ The
1
Optimizer(y(" (xg+t - Optimizer (n"(x"))
x'
180
The result is an unbiased estimate of true KG(x) and scales better to higher dimensional X, the univariate Z is discretized by Monte Carlo samples instead of X. However, for a good estimate, n, must be large, many Opt aptimizer( () calls are required. See Figure centre right.
191
192
We instead propose simple mixture of the two approaches above that both scales to higher dimen- sional X and drastically reduces the number of Optimizer () calls.
194
Hybrid KG: given x=+1, following KG we use n, values of Zj. construct 14; 1 (xx) and use Optimizer() () to find the peak location z/. The set of peak locations X {=i,.... 2,, is used as a dynamic optimized discretization in KGa thus analytically computing an extremely tight lower bound of the true KG(z). Let 4² = μ= (Xx) and similarly for o" (x"+ Hybrid KG given by
KG EEzlmxxxxx + 2""""==""""""""""""""""""""""""" +²'(x*+1)2
max y²"
195
Compared KGa, the hybrid method removes redundant points in the discretization Na. all X3 points contribute to max u=*1 (X.) and there are far fewer points. Compared to KG that samples many Z, and optimizes many xz. Hybrid KG uses far fewer Z, and optimizes far fewer zz. See [ for summary of evaluating ConBO with Hybrid KG. ConBO can be reduced to the REVI algorithm by replacing the dynamic importance sampled tasks with a pre-frozen discretization of tasks and the (dynamic optimized) hybrid KG with discrete KG over pre-frozen discretization of inputs. These changes drastically improve scalability enabling ConBO to be applied to a far broader range of applications.
197
Algorithm|
199
201
202
203
To ensure asymptotic convergence, in a discrete domain, we require that the acquisition function non-negative, (x)) 2 0, and the acquisition function is zero where GP variance is zero, (x) = 2 k- [z, x) 0. Therefore, always choosing -=+1 = arg max KG, (z) ensures only points with GP variance will be revisited until all points have no variance i.e. the true function is known for all points. We can ensure these properties by setting n, 2 2 and at least one Z, is equal to zero
205
207
209
Theorem Let 2 2 and let Z = [2 = 1, 1,.,, n2). IfO E Z then KG,(+) 2 f all x X and if is sampled infinitely often KG (z) = 0.
211
Proof is in the SM 1. The Z, values can be fixed, for n2 = 5 we use equal Gaussian quantiles Z = (0.1), - (0.3). ..., o 00((0.9} where (-) is the Gaussian CDF. Using quantile spacing
212
and odd n, ensures Z, Eb (0.55) = 0 is included which satisfies the assumptions of asymptotic convergence. See Fig (right).
213
The computational complexity of single call to ConBO requires the posterior variance (O(n²) and n.H, runs of Optimizer( (). Let Wealls be the number of times that Optimizer () calls the posterior 217
215
mean costing On)). Thus, ConBO total complexity is O(n² + mmaulls n. Note this is linear in
ng the size of the (small) dynamic optimal discretization over S x X. Thompson sampling with

---

Branin- Uniform
Triinle
Resenbrock- Unfiform
Rosenbrock-Triang
VTS
10
Can2D4
10
20
30
N 50 N 50 20 30
40
40 50
20 20
40 50
Figure 3: Opportunity Cost across range of synthetic test problems. The dummy baseline, KNN, worst all cases, policy gradient better however the Gaussian process based methods all perform better. UNI and El are not conditional algorithms yet outperform PEQL Amongst other conditional algorihtms, MTS, REVI and ConBO methods all perform significantly better. ConBO-3 outperformed by ConBO-5 demonstrating the improvement with more accurate KG,,
discretization uses one operation that scales cubically Onn"(n,n, n(n.n,) )² + (n.n,)) )²) in 220 the worst case and to reduce cost special techniques are required e.g. Fourier features, CG matrix inversion.
Experiments
222
We consider synthetic benchmarks and the three applications described in Section 1 In SM 4 we also present parallclization batch sampling results, and experiments with contextual, multi-task and multi-fidelity methods in SM 6. We also briefly test Hybrid KG for global optimization, SM 5, and observe that KG performs worse in the same computation time hence we exclude KGMC methods from the conditional experiments. For each benchmark, for evaluation, held out test tasks are sampled from P| P[s] x (s)), for each test task, s!nat, the predicted optimal input is computed, x (sfeat) and report the true black box output averaged over all test tasks, SM for details.
224
226
228
230
3.1
How Accurate is Hybrid Knowledge Gradient?
231
The theoretical Knowledge Gradient cannot be analytically computed and must be approximated (similarly, entropy based methods are not fully tractable). To illustrate the quality of the approxima- tions, we collected 20 points from the Rosenbrock test function and Gaussian process. Another point was then randomly selected and true KG was estimated by the different methods, keeping the 20 points fixed and only varying the nz parameter. Each estimate of KG was recalculated 50 shows the mean and two standard deviations. Discrete KG results in extremely loose, volatile bound. The (state of the art) MC method with n. = 50 has a cun-to-run variance of 135%, variance reduction techniques used in MC++ (latin hypercube inverse sampling and control variates) can reduce this to +6%. Hybrid KG with ni is extremely stable, run-to-run variance too small show, and is within the error margins of MC++ with n. 50 which is 17x more expensive to compute. Increasing n, tightens the bound, hybrid KG with n. = 5 has 98.2% of the value of n. 50. As result, Hybrid KG is much easier to optimize, the Adam optimizer may be used with much higher learning rate and lower momentum and thus converges much faster.
232
235
235
times and Table [
237
239
241
243
enhances the optimization of KGMC- By As aside, the recently proposed one-shot KG 34
245
freezing the Z values between calls to KGMC thus X may be reused, this enables joint gradient ascent of (*1+1 X*). In the global optimization use case (a constant single task) one-shot KG and Hybrid KG can be combined. In the conditional setting, the X; of past sampled tasks may be saved, however each call to ConBO must sample new tasks that are not in the saved history, one-shot KG style joint optimization is not possible. In our implementation we utilise caching of X; from old tasks to heuristically warm start finding X for new tasks, see SM 3.
248
249
250
3.2
Synthetic Functions
We perform low-dimensionaa toy experiments in an ideal setting sanity check where we expect all conditional methods to perform similarly. We use the popular Branin- Hoo and Rosenbrock test functions in 2D defining the (task, input) domain displayed in Figure]
253
7

---

Validation Error
Val. Em.. at
Acquisition Overhead (s)
100
93.6
93.6
33.4
NT5
934
934
60-
93.2
40
ConSO-3
9322
93.2-
ConBo
93.0
92.8
93.0
93.0
20
N 100
REV
REM
MT5
MTS 00
cons0
comti-3 D0
CunB0s
ComBi- Do
+
RPV
REM D0
MTS
MTT 0n
Cons0
coetto-3 po
cunsoS
comBO-- Do
N
100
Figure 4: Left: validation accuracy. Centre-left: validation accuracy after 50 samples. Centre-right: validation accuracy after 100 samples. Right: algorithm overhead in seconds. After 50 samples, none of the multi-task models outperform the baseline, El Tr (dashed line) suggesting all datasets can use similar hyperparameters. For the larger budget 100, all models outperform the baseline by 0.1% suggesting that for more fine-tuning, each dataset requires different hyperaparameters. In all cases, performing data optimization significantly increases performance.
255
Synthetic Functions Using the Rosenbrock and Branin Branin-Hoo functions, we consider a uniform and a triangular task weighting (s). For baselines, we adopt two policy based methods. KNN: (dummy baseline) randomly collect data, 2 N (s) takes a task, s, and returns the best input from 10 nearest neighbor tasks. PG: policy gradient, a parametric quadratic policy (s) = To(s) ()) is learnt by maximising observed performance values, each iteration samples a task from P(s) xx W (s) then is sampled with an e-greedy strategy. For controlled ablation study, all the BO methods fit the same GP and for evaluation use the same definition of z-"V (s) (Equation 3 methods only differ by their data acquisition strategy. UNI: random data collection, the most recent conditional methods PEQI, RE MTS with all parameters given in SM 3. ConBO-ng: given (s, æ). 20 tasks are importance sampled, KG,, with n, 3, 5 points is used. EI: expected improvement that treats (s, 2) inputs to
256
257
259
261
263
285
be optimized.
266
Results are shown in Figure 3 Policy based methods KNN and PG consistently perform worse than the Gaussian process methods. Surprisingly, the conditional BO algorithm PEQI performs similarly to UNI and much worse than El. All other conditional methods outperform all non-conditional methods.
267
289
3.3
CNN Training Hyperparameters
270
We apply MTS, REVI, and ConBO variants, and we adopt the recently proposed kernel used for BO with Common Random Numbers [35].
271
(((s, (8', o6 M x': + oe (f +3)).
where (z, 2z; () is a Matérn J kernel with length scales L. The first term models a common trend function across all tasks and the second term models how each task independently differs from the composed of another Matérn and the constant kernel to model global offset e.g. one dataset may have universally higher validation accuracy. This kernel has far fewer parameters than full multi-task product kernel, it is easy fit and scales an arbitrary number of tasks (or datasets) without adding extra parameters.
273
trend. The differences
275
277
In this problem setting, learning hyperparameters over similar datasets, one may expect that the optimal hyperparameters would the same for all datasets. Therefore, as baseline we apply El to learn the hyperparameters of the first dataset (task 1). We then evaluate the objective function (validation accuracy) on the rest of the datasets using the best observed hyperparameters from dataset 1, refer to this as EI + Transfer.
279
281
283
A Aggument Optimization versus Data Optimization In continuous task settings, it is not possible to evaluate every task. In discrete task settings with large sampling budgets N > [S], a user may desire a single high (although stochastic) output value, maxy, for every task. For example, in network hyperparameter optimization, the network with the best validation error, max y, will be deployed (and the hyperparameters x may or may not be reused). We refer to this data optimization (DO).
284
285
286
287
For simulated environments, the input (or "action") that generalizes providing the best long-term average performance, max, E[f x)), is deployed and we refer to this argument optimization
289
(AO). Past work [22] has shown that argument optimization finds inputs that generalize better but
8

---

Journey Times
6.1950
-70
6.1925
-75
6.1900
-80
0.1875
0.1850
KNN
-05
MTS
REVI
100
-105
100
Figure 5: Left: average journey times across a range of cities. Right: average profit across range of warchouses. ConBO-5 and El perform best these benchmarks.
291 may not provide optimal max y for tasks during the optimization run. The authors propose a "DO trick",
AO method for N - |||] iterations, then finally allocate sample per task with input, n+1 determined by El within the task. We apply this trick to all algorithms in this experiment.
293
294
Results are shown in Figure For the medium budget of 50 samples, ConBO performs best of the standard algorithms yet is still worse than the El+Tr baseline. Applying the final round of DO improves all results to match the baseline. For the large budget of 100 samples, all methods outperform the baseline suggesting dataset specific fine-tuning of hyperparameters is required to achieve best results. Again, DO provides significant boost performance for all methods.
295
297
299
Gaussian process kernel parameter learning required upproximately 2-5 seconds using Tensorflow. In Figure 4 (right) we show the runtime of each algorithm excluding model fitting and network training. purely acquisition function optimization time. MTS and ConBO-3 are quickest while Conbo-5 increases linearly over ConBO-3 and REVI takes much longer.
300
301
302
303
3.4
Ambulances and Warehouses
We apply all methods from Section 3.2
benchmarks from the www.tt. Simmpt.org library for simulation optimization problems. The ambulance problem (AMB) is 8-dimensional and consists of range of cities and one must optimize ambulance locations for each city. The Assemble-to-ordee problem (ATO) is 9-dimensional consisting of a range of warchouses and one must optimize target
305
307
308
stock level for each warchouse. Results are shown in Figure 5 Of the policy based methods, PG performs poorly and does not show on the plots whilst KNN performs poorly on AMB and performs
309
310
well on ATO suggesting that AMB is more difficult problem. Of the GP based methods, El performs well for smaller budgets. Although it is not a conditional algorithm we include it to highlight that sometimes the simplest idea can also work. Of the conditional methods, MTS, REVI, and ConBO-3
312
all perform similarly, either slightly (AMB) or largely (ATO) outperforming UNI. These methods struggle in higher dimensions while ConBO-5 uses more accurate acquisition function and only method that consistently performs well across all problems. We hypothesize that these problems are more difficult than the synthetics and CNN and truly stress test conditional algorithms.
314
316
Conclusion
317
Potential Limitations and Broader Impact there are multiple ways in which ConBO may fail, in this work we have not investigated how ConBO or Hybrid KG suffers with poorly learnt Gaussian process hyperparameters. In many upplications, poorly chosen kernel unoptimized hyperparame- 321 ters can lead to poor performance and our proposed methods may be more sensitive or more robust to these failures than alternative approaches. We propose general purpose optimization algorithm and analysis, we do not currently see any immediate societal impact.
319
323
324
We investigate Conditional Bayesian optimization and propose ConBO. ConBO is designed from the ground up to fully exploit the structure of conditional problems, namely that optimizing one task helps optimize similar tasks. Hence every point should be collected to maximise the benefit of all tasks. However, this can lead to excessive computational cost, particularly higher dimensions.
325
327
Thus also propose Hybrid KG that mixes past methods to be both fast and scalable.

---

References
329
330
[1] Victor Picheny and David Ginsbourger. A nonstationary space-time gaussian process model for partially converged simulations. SIAM/ASA Journal Uncertainty Quantification, 1(1):57-78, 2013.
332
[2] Shinkyu Jeong. Mitsuhiro Murayama, and Kazuomi Yamamoto. Efficient optimization design method using kriging model. Journal of aircraft, 42(2):413-420, 2005.
334
335
[3] Jasper Snoek, Hugo Larochelle, and Ryan P Adams. Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems, pages 2951-29553
336
2012.
338
[4] Diariétou Sambakhé, Lauriane Rouan, Jean-Noël Bacro, and Eric Gozé. Conditional opti- mization of noisy function using a kriging metamodel. Journal of Global Optimization, 73(3):615-636 636, 2019.
339
340
341
[5] Najjia Anna Dong, David Eckman, Xueqi Zhao, Shane G Henderson, and Matthias Poloczek. Empirically comparing the finite-time performance of simulation-optimizatoo algorithms. In 2017 Winter Simulation Conference (WSC), pages 2206-2217. IEEE, 2017.
342
343
[6] Jing Xie, Peter I Frazier, and Stephen E Chick. Bayesian optimization via simulation with pairwise sampling and correlated prior beliefs. Operations Research, 64(2):542-553 2016.
345
Matthias Poloczek, Jialei Wang, and Peter Frazier. Warm starting bayesian optimization. In 2016 Winter Simulation Conference (WSC), pages 770-781. IEEE, 2016.
347
[8] Kevin Swersky, Jasper Snoek, and Ryan Prescott Adams. Multi-task bayesian optimization. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2013.
349
350
[9] Supratik Paul, Konstantinos Chatzilygeroudis, Kamil Ciosek, Jean-Baptise Mouret, Michael Osborne, and Shimon Whiteson. Alternating optimisation and quadrature for robust control. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018.
351
352
[10] Saul Joscano-Paalmrin and Peter I Frazier. Bayesian optimization with expensive integrands. Xrxiv preprint arXiv: 180300861,, 2018
354
[11] Michael Pearce and Juergen Branke. Bayesian simulation optimization with input uncertainty. IEEE, 2017.
356
In 2017 Winter Simulation Conference (WSC), pages
[12] Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff functions. In Geoffrey Gordon, David Dunson, and Miroslav Dudfk, editors, Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, volume 15 of Proceedings of Machine Learning Research, pages 208-214, Fort Lauderdale, FL, USA, 11-13 Apr 2011. JMLR Workshop and Conference Proceedings.
358
360
362
[13] Wei Chu, Lihong Li, Lev Reyzin, and Robert Schapire. Contextual bandits with linear payoff functions. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, pages 208-214. JMLR Workshop and Conference Proceedings, 2011.
363
364
385
[14] Supratik Paul, Michael A Osborne, and Shimon Whiteson. Contextual policy optimisation. arXiv preprint arXiv:
386
2018
387
[15] Ian Char, Youngseog Chung, Willie Neiswanger, Kirthevasan Kandasamy, Andrew Oakleigh Nelson, Mark Boyer, Egemen Kolemen, and Jeff Schneider. Offline contextual bayesian optimization. In Advances in Neural Information Processing Systems, pages
368
369
2019.
[16] Deng Huang, Theodore T Allen, William I Notz, and R Allen Miller. Sequential kriging optimization using multiple-fidelity evaluations. Structural and Multidisciplinary Optimization, 3285):3693882, 2006.
371
372
373
[17] Matthias Poloezek, Jialei Wang, and Peter Frazier. Multi-information source optimization. In Advances in Neural Information Processing Systems, pages 2289-4299, 2017.
10

---

375
[18] Kirthevasan Kandasamy, Gautam Dasarathy, Jeff Schneider, and Barnabás Póczos. Multi-fidelity bayesian optimisation with continuous approximations. In Proceedings of the 34th International Conference on Machine Learning- Volume 70, pages 1799-1808. JMLR. org, 2017.
378
377
378
[19] Aaron Klein, Stefan Falkner, Simon Bartels, Philipp Hennig, and Frank Hutter. Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets. In Aarti Singh and Jerry Zhu, editors, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, volume 54 of Proceedings of Machine Learning Research, pages 528-536, Fort Lauderdale, FI., USA, 20-22 Apr 2017. PMLR.
379
380
382
383
[20] R. Bardenet, Mátyás Brendel, Balázs Kégl, and Michele Sebag. Collaborative hyperparameter tuning. In International Conference on Machine Learning, pages 199-207, 2013.
384
385
[21] David Ginsbourger, Jean Baccou, Clément Chevalier, Frédéric Perales, Nicolas Garland, and Yann Monerie. Bayesian adaptive reconstruction of profile optima and optimizers. SIAM/ASA Journal on Uncertainty Quantification, 2(1):490-510, 2014.
386
387
388
[22] Michael Pearce and Juergen Branke. Continuous multi-task bayesian optimisation with correla- tion. European Journal of Operational Research, 27063):004-11055 2018.
389
[23] C. E. Rasmussen and C. K. I. Williams, Gaussian Processes for Machine Learning. MIT Press, 2004.
381
[24] D. R. Jones, M. Schonlau, and W. J. Welch. Efficient global optimization of expensive black-box functions. Journal of Global optimization, 13(4):455-493 1998.
393
[25] Andreas Krause and Cheng S Ong- Contextual gaussian process bandit optimization. In Advances in neural information processing systems, pages 2447-2455, 2011.
395
[26] Philipp Hennig and Christian J Schuler. Entropy search for information-efficcent global opti- mization. Journal of Machine Learning Research, (1UJun)):809-1883) 2012.
397
398
[27] J. Villemonteix, E. Vazquez, and E. Walter. An informational approach to the global optimization of expensive-to-evaluate functions. Journal of Global Optimization, 44(4):509-55) 534, 2009.
399
28 J. M. HernAZandez-LLoatt M. W. Hoffman, and 7. Ghahramani. Predictive entropy search for efficient global optimization of black-box functions. In Advances in Neural Information P Processing Systems, pages 918-926. Curran Associates, Inc., 2014.
401
402
403
[29]
Zi Wang and Stefanic Jegelka. Max-value entropy search for efficient bayesian optimization. In Proceedings of the 34th International Conference on Machine Cetrning-Vollume 70, pages 3627-3635. JMLR. org, 2017
405
[30] P. Frazier, W. Powell, and S. Dayaník. The showledge--grdiiee policy for correlated normal beliefs. INFORMS Journal on Computing, 21(4):599-613, 2009.
407
408
[31] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
409
[32] Warren Scott, Peter Frazier, and Warren Powell. The correlated knowledge gradient for simulation optimization of continuous parameters using gaussian process regression. SIAM Journal on Optimization, 2163):99610066, 2011.
411
412
413
[33] Jian Wu and Peter Frazier. Discretization-fret free knowledge gradient methods for bayesian optimization, arXiv preprint ToX.vv/777.065..2 2017.
414
415
[34] Maximilian Balandat, Brian Karrer, Daniel Jiang, Samuel Daulton, Ben Letham, Andrew G Wil- son, and Eytan Bakshy. Botorch: A framework for efficient monte-carlo bayesian optimization. Advances in Neural Information Processing Systems, 33, 2020.
416
417
418
[35] Michael Pearce, Matthias Poloczek, and Juergen Branke. Bayesian optimization allowing for common random numbers. arXiv preprint 191.9219,, 2019.
419
11

---

Checklist
420
421
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] ConBO is theoretically grounded, see Theorems 1-3. ConBO outperforms recently published works on a range of problems, see Section 3,
423
425
ConBO is easily parallelizable, see SM Section 4.
428
(b) Did you describe the limitations of your work? [Yes] see Section 4
(c) Did you discuss any potential negative societal impacts of your work? [Yes] see Section
428
429
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
430
2. If you are including theoretical results...
432
(a) Did you state the full set of assumptions of all theoretical results? [Yes] see SM Section
433
I
434
(b) Did you include complete proofs of all theoretical results? [Yes] see SM Section
435
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main exper- imental results (either in the supplemental material or as a URL)? [Yes] All code is submitted with the SM
437
439
(b) Did you specify all the training details (c.g.. data splits, hyperparameters, how they were chosen)? [Yes] all parameters for all baselines are given in SM section 3.
440
(c) Did you report error bars (e.g.. with respect to the random seed after running experi- ments multiple times)? [Yes]
442
443
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes] SM section
444
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
448
(a) If your work uses existing assets, did you cite the creators? [Yes] code from Simopt.org cited.
447
448
(b) Did you mention the license of the assets? [N/A]
449
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes] the benchmark test problems.
450
451
(d) Did you discuss whether and how consent was obtained from people whose data you're using/curating? [N/A]
452
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
454
455
5. If you used crowdsoureing or conducted research with human subjects...
456
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
457
458
(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
459
460
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
12


Reviews:
[{"review_id": "c24ixfD1bB1", "reviewer": "Reviewer_QbMn", "summary": "The authors tackle an underexplored problem they call conditional BO, which is a variant of multi-task BO in which one seeks to identify the optimum of each task, instead of the traditional multi-task BO seeks to identify the optimum of the average over all tasks. \n\nThe authors introduce ConBO, which uses the knowledge gradient acquisition function to determine the next point to evaluate, and discuss its efficient computation through a combination of intelligent sampling and discretization. \n\nConBO is evaluated on a variety of problems, including three good applications: hyperparameter optimization, ambulance base placement, and assemble-to-order policy selection. Results are promising, and suggest that ConBO outperforms a well-chosen set of sensible baselines.", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n* The applications were well suited to the problem at hand. \n* The discussion of prior work and baseline performance measures was very comprehensive. \n* The problem itself is both of theoretical interest and practical value. \n* I also appreciated the author’s efforts to speed up the acquisition function computation; this is too often ignored in the literature as of minimal importance. \n\nWeaknesses:\n* The primary weakness, which the authors themselves point out, is that this problem of conditional BO is nearly identical to the problem of Bayesian quadrature; though the acquisition function itself is novel to my knowledge, the problem formulation is not. \n* The acquisition function is the expectation of the KG acquisition over tasks s $E_s[ KG(s)]$, where the expectation is taken w.r.t measure W(s). My intuition says this acquisition seeks to efficiently compute $E_s[ x(s)]$ where $x(s) = \\max_s \\mu(x)$ w.r.t. W(s), which seems to be precisely Bayesian quadrature (please correct me if my intuition is wrong!) \n* Thus, any method for Bayesian quadrature ought to work well, unless I am missing something. There don’t seem to be any Bayesian quadrature methods in the baselines, is there a reason why? \n* I think this paper can be greatly strengthened if the authors define different measures of success. Obviously there is no way to identify exactly x(s) for all s, so the authors have decided their measure of success is $E_s[ x(s) ]$, which is one of the most logical. But there are other measures of success for conditional BO that do not necessarily involve this expectation; for example, computing accurately the bounds $\\max_s | x(s)|$. Perhaps these can be used to showcase the strengths of ConBO, though this would likely require significant revision. \n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "Z0Wmv90wjGG", "reviewer": "Reviewer_wodU", "summary": "This paper proposes a new algorithm for a problem entitled \"conditional Bayesian optimization.\" The goal is to search for a *series of input locations* that maximizes the performance of a given set of tasks. Unlike multi-fidelity optimization, the objective is to find good values for all tasks rather than the high-fidelity task. The main contribution is a Hybrid Knowledge Gradient (KG) algorithm that mixes two ways of computing KG: one that discretizes the input space (x, t) and computes KG for a discrete set of points,  and another one that uses the reparametrization trick to perform stochastic gradient descent over Monte Carlo samples. Formal guarantees about this method are also given. Empirical results show that the proposed method effectively reduces computation time and improves the quality of the solution.", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "**Strengths**\n\nI think this paper explores a problem formulation that is not well-study, but it could be relevant. It is a subtle variation of multi-task learning that focuses on multiple solutions (on for each task). This distinguishing is a little bit tricky, and I was a little bit confused for a while. Fortunately, the authors give three different examples that illustrate the setting that they are considering. I also enjoy the experiment section of this paper. The authors discussed the studied problems adequately, and I found the visualization and presentation of the results very interesting.\n\n**Weaknesses**\n\nOriginality. This paper would be more substantial if the main algorithmic contribution weren't so tight to computational tricks to compute Knowledge Gradient or if the problem under investigation wasn't similar to REVI's [22] formulation.  If I understood correctly, the main contribution is a speedup over REVI based on how KG is calculated. The theoretical analysis adds substance to the paper, but none of these results are ground-breaking.\n\nClarity. The presentation of section 2 could be a little bit more straightforward. There is unnecessary confusion about how information is captured by the inference procedure (the Gaussian process) and how decision-making is handled (construct an acquisition function). \n\n**Minors**\n\n- 115 - \"integrating over tasks multiplies the computational burden\" multiples -> increases\n- 130 - \"non-trivial modification to be able to account for how a sample affects similar tasks.\" Wouldn't a kernel that captures the correlation between similar tasks be enough to accomplish this?\n- 133 - location of the peak P(x | ...) -> mutual information between both distributions (or RV), you can remove the word peak here.\n- 149 - I don't think you meant to say that squared exponential or Matern is a kernel that factorizes here.\n- l89 - n_s? Shouldn't it be n_z? \n\n**After rebuttal**\n I appreciate the author's detail response and I would strongly recommend the authors to add this level of clarity and detail to the original manuscript. Thanks", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Yes.", "review_text": "", "extra_scores": {"ethical_concerns": "N/A.", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "4"}, "extra_text": {}}, {"review_id": "PGxHeJyMxBz", "reviewer": "Reviewer_BDnT", "summary": "The authors propose a method for conditional optimization problems: finding the optimum of each function in set of objective functions. In this setting, the method jointly selects the parameters to evaluate and the task to evaluate. The authors propose a theoretically-grounded variant of the Knowledge Gradient to address this class of problems and propose a new method of approximating the KG acquisition function. ", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Strengths:\n-\tElegant, theoretically-grounded formulation for conditional optimization\n  -\tBeing myopically optimal in the value of information framework is a nice property\n-\tThe hybrid KG results show that is a quite stable method for approximating KG.\n\nWeaknesses:\n-\tThe class of problems is not very well motivated. The CIFAR example is contrived and built for demonstration purposes. It is not clear what application would warrant sequentially (or in batches) and jointly selecting tasks and parameters to simultaneously optimize multiple objective functions. Although one could achieve lower regret in terms of total task-function evaluations by selecting the specific task(s) to evaluate rather than evaluating all tasks simultaneously, the regret may not be better with respect to timesteps. For example, in the assemble-to-order, even if no parameters are evaluated for task function (warehouse s) at timestep t, that warehouse is going to use some (default) set of parameters at timestep t (assuming it is in operation---if this is all on a simulator then the importance of choosing `s` seems even less well motivated). There are contextual BO methods (e.g. Feng et al 2020) that address the case of simultaneously tuning parameters for multiple different contexts (tasks), where all tasks are evaluated at every timestep. Compelling motivating examples would help drive home the significance of this paper.\n-\tThe authors take time to discuss how KG handles the continuous task setting, but there are no experiments with continuous tasks\n-\tIt’s great that entropy methods for conditional optimization are derived in Section 7 in the appendix, but why are these not included in the experiments? How does the empirical performance of these methods compare to ConBO?\n-\tThe empirical performance is not that strong. EI is extremely competitive and better in low-budget regimes on ambulance and ATO\n-\tThe performance evaluation procedure is bizarre: “We measure convergence of each benchmark by sampling a set of test tasks S_test ∼ P[s] ∝ W(s) which are never used during optimization”. Why are the methods evaluated on test tasks not used during the optimization since all benchmark problems have discrete (and relatively small) sets of tasks? Why not evaluate performance on the expected objective (i.e. true, weighted) across tasks?\n-\tThe asymptotic convergence result for Hybrid KG is not terribly compelling\n-\tIt is really buried in the appendix that approximate gradients are used to optimize KG using Adam. I would feature this more prominently.\n-\tFor the global optimization study on hybrid KG, it would be interesting to see performance compared to other recent kg work (e.g. one-shot KG, since that estimator formulation can be optimized with exact gradients)\n\nWriting:\n-\tL120: this is a run-on sentence\n-\tFigure 2: left title “poster mean” -> “posterior mean”\n-\tFigure 4: mislabeled plots. The title says validation error, but many subplots appear to show validation accuracy. Also, “hyperaparameters” -> hyperparameters\n-\tL286: “best validation error (max y)” is contradictory\n-\tL293: “We apply this trick to all algorithms in this experiment”: what is “this experiment”?\n-\tThe appendix is not using NeurIPS 2021 style files\n-\tI recommend giving the appendix a proofread:\n  - Some things that jump out\n    -\tP6: “poster mean”, “peicewise-linear”\n    -\tP9: “sugggest”\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "Limitations and societal impacts are discussed, but the potential negative societal impacts could be expounded upon.", "review_text": "", "extra_scores": {"needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}, {"review_id": "BjLs52HAMQq", "reviewer": "Reviewer_aWTB", "summary": "This paper presents a framework for conditional optimization, which is called ConBO. Authors also proposed a new acquisition function: hybird knowledge gradient, which is built on KG. Empirical experiments show their proposed method outperform many competing algorithms. ", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "I reviewed this paper at AISTATS. While I was glad to see that authors have added additional comparison experiments on multi-task and multi-fidelity methods (but only in the supplementary material, not int the main paper), a lot of goods points raised by reviewers there are still not addressed in this version. \n\nimprovements:\n1. many related work are covered in this version and this paper is well-organized\n\nweaknesses:\n1. Some related work is still missing.\n2. There are insufficient updates in the main paper. Compared with the last version, all figures and empirical evaluation are exactly the same, without any changes. Although there are some extra experiments in the supplementary material.  Notably the comparison with [1] is still missing. This work is closely related to [1] and a comparison with [1] is recommended.\n3. Given the proposed method doesn't provide sufficient improvements over KG, I would have expected stronger and extensive empirical evaluation. \n\n\n\n\n\n\n[1]. Krause, A., Ong, C.S.: Contextual Gaussian Process Bandit Optimization. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P.L., Pereira, F., and Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 24. pp. 2447–2455. Curran Associates, Inc. (2011).\n\n", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "See the above review.", "review_text": "", "extra_scores": {"ethical_concerns": " ", "needs_ethics_review": "No", "ethics_review_area": "", "time_spent_reviewing": "2"}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper addresses a framework for conditional Bayesian optimization with a newly proposed hybrid KG. The method seems to be a variant of multi-task BO, where the optimum of each task is determined in contrast to the multi-task BO. The applications well suited to the problem are described to demonstrate the usefulness of the method. Reviewers agree that the paper is of theoretically interest and has practical value as well. However, a few issues were raised, details of which can be found in the reviewers' comments. During the discussion period, none of the reviewers were willing to champion this work. The paper still needs more substantial updates in particularly experiments. 


Decision: Reject