SnAKe: Bayesian Optimization via Pathwise Exploration
Anonymous Author s)
Abstract
Bayesian Optimization is a very effective tool for optimizing expensive black-boxfunctions.
Inspired by applications developing and characterizing reaction chemistry using droplet microfluidic reactors, we consider a novel setting where theexpense of evaluating the function can increase significantly when making large input changes between iterations.
We further assume we are working asynchronously,meaning we have to select new queries before evaluating previous experiments.
This paper investigates the problem and introduces ‘Sequential Bayesian Optimization via Adaptive Connecting Samples’ (SnAKe), which provides a solution byconsidering large batches of queries and preemptively building optimization pathsthat minimize input costs.
We investigate some convergence properties and empirically show that the algorithm is able to achieve regret similar to classical Bayesian
Optimization algorithms in both the synchronous and asynchronous settings, whilereducing the input costs significantly.
We show the method is robust to the choiceof its single hyper-parameter and provide a parameter-free alternative.
Introduction
We introduce a method for black-box optimization that keeps input variations as small as possible.
A black-box function is expensive to evaluate (with respect to time or resources) and we do nothave access to gradients.
Classically, black-box optimization finds an optimum by sequentiallyquerying the function.
We study a variation of this problem, with two important differences.
First,we introduce the idea that large changes in inputs, between iterations, cause the function to becomemore expensive to evaluate.
Second, we do not assume observations are available immediately: delaybetween querying the function and getting a result leads to asynchronous decision making.
As a motivating example consider a droplet microfluidic reactor [44] (see Figure 6 in Appendix).
Insuch a reactor, we can quickly pump in chemicals, expose them to certain conditions and collectthe results of our experiments as they leave.
However, large changes in temperature mean that thereactions are no longer in steady-state and this makes the evaluations unreliable until the systemstabilizes.
Smaller changes mean that we never leave steady state, or that the system is easier tostabilize.
Further, we have to wait for droplets to exit the reactor before obtaining observations.
More broadly, consider optimization requiring spatially continuous exploration.
Black-box optimization has been investigated for optimal placement of pollution sensors [15], and machine learningmethods for detecting the sources of pollution have been developed [28].
Mobile pollution readers,such as those used by AirQo in Uganda [ https://www.airqo.net/about ], would face similarchallenges if black-box optimizers were used to track the pollution sources.
Samaniego et al.
[36]show a more concrete example where autonomous vehicles use black-box optimizers to monitor andtrack water contamination sources in the Ypacarai lake in Paraguay.
Classical Bayesian Optimization (BO) [18, 37] provides effective solutions to black-box optimization.
However, typically BO follows a myopic approach, in that BO chooses the next query based only onthe current state of the surrogate model.
Classical BO reduces uncertainty in unexplored areas andthen quickly returns to promising areas with no regard for the distance between consecutive querypoints in the input space.
This means that BO will incur very high input costs .
However, having zero changes in input space is obviously not a good solution.
After all we want toexplore the search space to find the optimum point.
We seek an algorithm that preserves the essenceof Bayesian Optimization.
Consider a scenario where we know a large number of inputs we wantto query.
In this case, we could simply order the queries to attain the smallest input cost.
A goodsolution would simply require selecting a large number of queries, creating an ordering, and thensimply following the path defined by the ordering.
Once we start obtaining new information, wecould update our beliefs and update our optimization path.
This paper proposes S eque n tial Bayesian Optimization via A daptive ( K )Connecting Sampl e s (SnAKe).
Just as the snake grows by carefully eating items in the classic arcade game, a SnAKe optimizationpath grows from carefully adding queries to the evaluation path.
Related Work
‘Process-constrained Batch Bayesian Optimization’ [45], is the closest analog to our setting of BOwith input costs.
Vellanki et al.
[45] navigate the physical limitations in changing the input spaceby fixing the complicated inputs for every batch.
Rather than fixing the difficult-to-change inputs,we penalize large input variations in line with the costs (time and resources) of changing conditions.
SnAKe decides when to make expensive input changes.
Waldron et al.
[46] compare the use oftransient variable ramps (small input changes in a pre-determined manner) against a full steady-statedesign of experiments approach for learning parameters of chemical kinetic models.
They show thatthe transient approaches give less precise estimates but much faster.
SnAKe combines the best ofboth worlds, automatically designing experiments while keeping input changes small.
‘Cost-aware Bayesian Optimization’ [26, 29, 39] optimizes the regret with respect to cost per iteration.
These costs, which are fixed throughout the BO process, arise when some regions of the input domainare more expensive to evaluate than others.
Similar ideas appear in Multi-fidelity BO [19, 33] wherecheap approximations for the objective function are used.
Diverging from prior works, our settingdoes not assume that different regions incur different costs.
Instead, SnAKe addresses the settingwhere the difference between adjacent inputs causes the costs to change.
The query order changes thecost, therefore we focus on optimization paths instead of optimization sets.
The baseline of Cost-aware BO, Expected Improvement per unit Cost (EIpu), EI( x ) / Cost( x ) , is notdirectly applicable to our setting because the cost of not changing inputs, i.e. choosing x t = x t − 1 , iszero.
We can instead use:
[Equation]
Three main issues: (a) Eq.
(1) introduces new hyper-parameter γ with no obvious way to choose it,(b) Eq.
(1) effectively penalizes the acquisition function far away from the data, so near local optimawe only penalize exploration and may over-exploit, (c) Asynchronous extensions have a tendency toencourage batch diversity, so we introduce a new trade-off between penalizing locally and far away.
As we shall see, SnAKe shows robust results with just one hyper-parameter.
Look ahead Bayesian Optimization [10, 13, 24] considers possible future queries to make our currentchoice less myopic.
This would be a very great way of selecting the inputs to query, as we couldsimply look-ahead at what any classical BO would choose and then order the queries accordingly.
However, one is only able to look ahead for a few iterations, due to the computational complexity oflooking too far ahead into the future [5, 48].
For example, [27] combines Markov Decision Processes[34], look-ahead and cost-aware ideas, but is limited by the short time horizons rollout can handle.
Reinforcement Learning [43] can be used with longer time-horizons, but it requires access to asimulated environment to train.
A computationally cheaper alternative which allows us to select many inputs to query is Batch
Bayesian Optimization (BBO) [2, 12].
BBO is the setting where we are able to parallelize functionevaluations, and as such we want to select multiple queries simultaneously.
González et al.
[13] and
Jiang et al.
[17] link BBO with look-ahead BO, using a Local Penalization method [12] and Expected
Improvement (q-EI) [11] respectively.
Unfortunately, they restrict themselves to smaller batch sizes( q ≤ 15 ) due to computational expense.
Asynchronous Bayesian Optimization [1, 20] addresses theproblem of choosing queries while waiting for delayed observations.
Methods3.1 Problem Set-up
We consider finding the maximum, x ∗ = arg max x ∈X f ( x ) , of a black-box function, f , where X isa compact subset of R d .
We assume f is continuously differentiable and expensive to evaluate.
Weseek the optimum point while keeping the number of evaluations small.
We evaluate the functionsequentially, over a discrete and finite number of samples, t = 1 , ..., T .
For every query, x t , weobtain a noisy observation of the objective, y t
= f ( x t ) + η t , where η t ∼ N (0 , s 2 ) is Gaussian noise.
We assume there is delay, t delay , between choosing a query and getting an observation.
So ourdata-set at iteration t is given by D t
= { ( x i , y i ) : i = 1 , ..., t − t delay − 1 } .
If we set t delay = 0 , werevert to classical sequential Bayesian Optimization, otherwise we are in an asynchronous setting.
Finally, we assume there is a known inherent cost to changing the inputs to our evaluation, C ( x t , x t +1 ) .
We will be using simple regret, SR t = f ( x ∗ ) − max i =1 ,...,t f ( x i ) as the performance metric.
Wewant to minimize regret while keeping (cid:80) T − 1 t =1 C ( x t , x t +1 ) small.
As is common in Bayesian Optimization, we will model the black-box function by putting a Gaussian
Process (GP) prior on f ∼ GP ( µ 0 , σ 20 ) .
Since we have Gaussian noise, the posterior, f | D t is also a
GP, whose mean function, µ t ( · ) , and covariance function, κ t ( · , · ) , can be calculated analytically [35].
3.2 General Approach
For our general approach, we will seek to create a large batch of queries that we want to evaluate,and then plan-ahead a whole optimization path.
This is useful for three reasons: (1) it allows usto order the queries in a way that reduces input cost.
(2) It allows us to deal with any delay ingetting observations, because we can pre-select future queries.
(3) It remains computationally feasibleto plan-ahead even for very large time horizons.
We can follow this ordering or path until newinformation is available, after which we will update our path.
While we will provide a specific way of creating, planning and updating the paths, we note that thegeneral ideas can be extended to suit different settings.
As an example, in Appendix C we show howto alter SnAKe to optimize multiple black-box functions simultaneously.
3.3 Creating a Batch Through Thompson Sampling
We need to produce batches that are representative of the current state of the surrogate model.
Notonly this, but the method should allow for big batch sizes, since we want to produce batches as big asour budget (which is usually much larger than the batch size most methods consider).
For example,in a micro-reactor, we might be interested in batches that contain hundreds of points [44].
Kandasamy et al.
[20] offers a promising solution where every point in the batch is independent.
Themethod is based on Thompson Sampling, which uses the GP’s own randomness to create a batch.
Asingle batch point is chosen by drawing a realization of the GP, and optimizing it.
The queries willfill out the space, and they are more likely to be on promising, and unexplored areas.
It should workvery well in our context given we expect our initial batch sizes to be very large, so the sample shouldbe representative of the current state of our surrogate model.
3.4 Creating a Path via the Travelling Salesman Problem
After selecting a batch of queries, P t = { x ( i ) t } ˜
Ti =1 , we order them.
We do this by embedding a graphinto the batch, where the edge weights are the cost for changing one input to another.
We then findthe shortest path that visits every point, i.e., solve the Travelling Salesman Problem (TSP) [4, 6].
Section 3.9 discusses the computational cost.
Mathematically, we define the graph G =
( V, E, W ) , with V =
{ i ∈ 1 , ..., ˜ T : x ( i ) t ∈ P t } ,
E = { ( i, j ) : i, j ∈ 1 , ..., ˜ T } , and W = { w ij =
C ( x ( i ) t , x ( j ) t ) :
( i, j ) ∈ E } , where ˜ T is the numberof batch samples.
We solve the TSP in G to obtain our latest optimization path.
A simple examplewould be to try to minimize the total distance travelled in input space, by selecting the Euclideannorm as cost C ( x t , x t +1 )
=
|| x t − x t +
|| .
3.5 Naively Updating the Optimization Path
After updating the GP with new observations, we want to use this information to update our path.
Wepropose updating our strategy by creating a new batch of points.
At iteration t , the remaining budget has size T − t .
We first propose sampling T − t queries through
Thompson Sampling, and then solving the Travelling Salesman Problem.
However, we show thisleads to the algorithm getting ‘stuck’ in local optima.
This is because every time we re-sample, wenaturally include some exploitation in the batch, and this exploitation will always be the next pointchosen by the TSP - we will never reach the exploration algorithm steps.
Figure 1a shows an example.
3.6 Escape Analysis
To try and solve the convergence problem introduced by naive resampling, we briefly analyze it.
This analysis assumes we receive noise-less observations.
However, all sampling can be done in thepresence of noise by calculating the corresponding posterior.
We focus on Thompson Sampling.
Definition 3.1.
(Thompson Sample)
We say x ( i ) t is Thompson Sampled, and we write x ( i ) t ∼ τ t , if:
[Equation]
i
[Equation]
t
[Equation]
max x ∈X f ( i ) t ( x ) where f ( i ) t ( · ) ∼ GP ( µ t , κ t | D t )
Note that the sampling distribution changes at each iteration.
One particular concern, diagrammed in Figure 1a, is the possibility of the method being stuck in acertain area.
Let B δ ( a ) be a Euclidean δ -ball centered at a .
Assume further that x t − 1 ∈ B δ ( a ) .
Definition 3.2.
(Non-escape Probability)
We define the non-escape probability, p t , at time t , as theprobability of a Thompson Sample falling into B δ ( a ) .
That is, for x ( i ) t ∼ τ t :
[Equation]
Let P t =
( x (1) t , ..., x ( T − t ) t ) ∼ τ t i.i.d. .
Of particular interest to us, is the number of ‘non-escapes’ inthe sample, N t = |P
t ∩ B δ ( a ) | .
We are guaranteed to escape if no sample falls in B δ ( a ) , so we saywe have fully escaped if x ( i ) t / ∈ B δ ( a ) ∀ i .
Remark 3.3.
Note that N t ∼ Binomial ( T − t, p t ) as all the samples are mutually independent,therefore the probability of fully escaping is P ( N t = 0) =
(1 − p t ) T − t .
This means we can onlyexpect to fully escape if p t is very small.
Therefore we are interested in the behavior of p t as we gain more and more information about f in
B δ ( a ) .
We now consider the circumstances under which p t becomes very small:
Areas without stationary points: Consider the case when our objective f does not contain astationary point in B δ ( a ) .
Then the maximum of f on the closure of the ball, ¯ B δ ( a ) , must lie onthe boundary of the ball.
In particular, if we assume that our GP model has no error in B δ ( a ) , andassuming continuity of sample paths, then it must be p t = 0 .
Intuitively, the area itself contains enough information to ensure, with complete certainty, that theglobal optimum does not lie in the area.
We hope that, as we collect information inside areas withoutstationary points, p t → 0 , and we will eventually leave them with small probability of returning.
Appendix D.2 shows an example of this happening very fast.
Areas with stationary points: Areas with stationary points pose a much bigger problem.
We willrestrict our arguments to local maximums since this is where we have observed the problem.
Assumethat f has a local optimum in B δ ( a ) , higher than any other we have observed before.
In this case,any sample taken from a Gaussian Process with no error in B δ ( a ) will have a local maximum inside
B δ ( a ) , and therefore it is possible that this local maximum is the global solution.
As we increase the information inside the area, p t is not guaranteed go down to zero.
This makesintuitive sense; the only way of knowing if a local optimum is not a global optimum is by samplingaway from it - therefore with limited information we will allocate a certain probability to the globaloptimum being inside B δ ( a ) .
We include a clear example where p t → p > 0 in Appendix D.1.
Sampling consistently in a promising area is not necessarily a bad thing, indeed we want to exploitnear possible global optimums.
However, the question then becomes, how long will it take us toleave?
Recall the probability of fully escaping is (1 − p t ) T − t , and therefore it will be increasing as tincreases, even if p t is (almost) constant.
Remark 3.4.
Assume that p t → p > 0 , and that we have a high escape probability after t e iterations,i.e. (1 − p ) T − t e is large, leaving us T − t e iterations to explore the rest of the space.
If we increase ourbudget from T to T ′ , we will not have a high probability of escape until (1 − p ) T ′ − t
=
(1 − p ) T − t e ,i.e. t = T ′ − T + t e .
This leaves us with T ′ − T ′ + T − t e = T − t e iterations to explore the rest ofthe space.
Increasing our budget does not increase our budget after leaving B δ ( a ) !
It only means wewill be stuck in B δ ( a ) for a longer time.
This is very concerning as the method will be very myopic;if it finds a local optimum, it is likely that it will spend a very large amount of the budget exploiting it.
3.7 Escaping with ϵ -Point Deletion
Intuitively, the convergence problem stems from the constant resampling, because every time were-sample we are reintroducing a number of exploitative points.
This forces us to require N t = 0to be able to fully escape.
As a solution, we propose sampling more points than we need to geta measure of the excess exploitation, and then deleting batch points that are similar to previouslyexplored points.
Algorithm 1, which we dub ϵ -Point Deletion, still allows us to exploit local optimaif we sample many points near them, however, it should eventually move on.
Note that Point Deletion uses the Euclidean norm, and it is independent of the cost function.
This isbecause we are trying to escape local minima of simple regret.
Algorithm 1 ϵ input: New proposed batch P t (size T ), set of already queried points Q t (size t ), and deletion distance ϵ for x ∈ Q t do ˜
d ← min x ′ ∈P t || x − x ′ ||
if ˜ d < ϵ then # find the closest point to the query x in the new batch ˜ x ← arg min x ′ ∈P t || x − x ′ || else # else pick a random sample
˜ x ← Random ( P t ) end if # remove said point from the batch
P t ← P t \ { ˜ x } end for output: A batch P t (size T − t )
ϵ -Point Deletion allows us to escape local optima by directly increasing the probability of fullyescaping without changing p t .
To see this, consider the case where there are q t previously queriedpoints inside the ball, B δ ( a ) , and set ϵ ≥ 2 δ .
It follows that we will escape if N t ≥ q t , which ismuch better than requiring N t = 0 .
In particular, consider the expected number of ‘non-escapes’, E [ N t ] = p t T .
Around a local maximum,this quantity may be approximately constant with time, p t T ≈ pT , so we can reasonably expect anescape when q t = pT .
This is a desirable property because we still want to allow exploitation ofpromising areas.
However, this time we leave T − q t =
T (1 − p ) extra iterations to explore theremaining space!
Increasing the budget will benefit both the exploitation and the exploration insteadof only the former (in contrast with remark 3.4).
Figure 1 gives an empirical example where we use Point Deletion, with ϵ = 0 . 1 , to escape a localoptimum.
We observe the expected behavior from our brief analysis.
For Point Deletion, we calculate the escape prediction as pT ≈ 74 , using ˆ p ≈ 0 . 74 , which we estimated in Appendix D.1.
We can seethat without Point Deletion, we remain stuck in the first local optimum.
[Figure]
3.8 SnAKe
Algorithm 2 which we dub ‘Sequential Bayesian Optimization via Adaptive Connecting Samples’(SnAKe), combines the ideas of previous sections.
Figure 2 diagrams the most important steps of
SnAKe.
Section 4 develops an effective, parameter-free alternative to the choice of ϵ .
Note there is no requirement for data to be available immediately following querying.
If t delay > 0 ,we can simply stick to the latest path.
It works without modification on the asynchronous setting.
This is vital since we were inspired by chemical experiment design which can exhibit asynchronicity.
Algorithm
input : Optimization Budget, T .
Deletion constant, ϵ .
begin : Create initial batch, P 0 , uniformly.
Choose starting point x 0 .
Q 0 ← { x 0 } .
Create initial path, S 0 , by solving TSP on P 0 .
for t = 1 , 2 , 3 , ..., T do
Check if any running evaluations are finished if there are new observations then
Update surrogate model
Create batch of size T using Thompson Sampling, P t −
P t − 1 ← ϵ -Point Deletion
( P t − 1 , Q t − 1 ) ˜
S ← TSP( P t , source = x t − 1 ) \
x t − 1 }
S ← Q t − 1 ∪ ˜ S end
if
Choose next query point from schedule: x t ← S t
Q t ← Q t − 1 ∪ { x t }
Evaluate f ( x t ) end for
3.9 Computational Considerations
We use Wilson et al.
[47] to efficiently sample the GP, the samples are optimized using Adam [21].
The Travelling Salesman Problem’s solution is quickly approximated using Simulated Annealing[22], and we use an adaptive grid to reduce the number of samples away from our current input.
Withthese modifications, we were able to comfortably run all experiments in a CPU (2.5 GHz Quad-Core
Intel Core i7).
More details can be found in Appendix E.1.
Experimental Results
For all experimental results we report the mean and the standard deviation over 25 experimental runs.
We give the full implementation details and results in Appendix E and F respectively.
All classical
BO methods are implemented using BoTorch [3] and GPyTorch [9].
[Figure]
In all experiments, we examine SnAKe for ϵ = 0 , 0 . 1 , and 1 .
We further introduce a parameter-freealternative by adaptively selecting ϵ to be the smallest length scale from the GP’s kernel, and denoteit ℓ -SnAKe.
SnAKe proves to be robust to non-zero choices of ϵ , we conjecture this happens becausealmost all excess exploitation is concentrated on a very small area.
For ϵ =
we observe very lowcost, at the expense of some regret.
Across all experiments we can see how SnAKe can traverse the search space effectively, constantlyachieving good regret at low cost.
While other cost-aware methods may also achieve good performance, they are very inconsistent suggesting they require careful tuning.
SnAKe achieves good,robust results with a single hyper-parameter.
We do highlight that SnAKe performs best when thereare enough iterations for the samples to adequately fill the search poor performance Perm10D by ordinary performance low-budget experiments
Appendix F)
[Figure]
[Figure]
[Figure]
[Figure]
[Figure]
4.1 Synthetic Functions
Sequential BO
This section examines the performance of SnAKe against Classical BO algorithms.
We compare Expected Improvement (EI) [31], Upper Confidence Bound (UCB) [41], and Probabilityof Improvement (PI) [23].
We compare against Expected Improvement per Unit Cost [39] asintroduced in Eq.
(1), with γ = 1 , and with Truncated Expected Improvement [36].
We alsointroduce a simple baseline, where we create a random Sobol sample, and then arrange an ordering by solving the TSP, and never update the path again.
We do this for three classical benchmark functions,and six total in the Appendix F.1.1.
We set the cost function to be the 2-norm distance between theinputs.
Figure 4 shows the results in the top row.
Asynchronous BO We explore the asynchronous setting, comparing Local Penalisation with UCB(UCBwLP) [1, 12], Thompson Sampling (TS) [20], and the same Random baseline from the sequentialsetting.
We also test on EIpu with Local Penalisation (EIpuLP) [Figure] with γ = 1 .
The results are shown inlast two rows of Figure [Figure] [Figure]
4.2 Reaction Control on SnAr Benchmark
We test our method on a real-world, SnAr chemistry benchmark [16].
We control four variables;equivalents of pyrrolidine, concentration of 2,4 dinitrofluorobenenze at reactor inlet, reactor temperature, and residence time.
We assume changing temperature, concentration and residence time incur aninput cost, owing to the response time required for the reactor to reach a new steady state.
We assumethe reactor as a first-order dynamic system, where the response to changes in input is given by:
[Equation]
Where s denotes the time after experiment x t is finished, ( x t ) i denotes the i th variable of the t thexperiment, (∆ x t ) i =
( x t +1 )
i − ( x t ) i , and α i is the system time constant.
We assume the systemreaches steady state once | ( x s )
i − ( x t +1 )
i | ≤ β i .
For input changes smaller than β
i , we assume alinear cost, defined by a parameter γ i .
Combining with the response time from (3):
[Equation]
[Equation] [Table] We assume that we can change variables simultaneously, so the total input cost is simply the longestresponse time within a given set of input changes.
That is, C ( x t , x t +1 )
= max i ∈
I
c
C
i ( x t , x t +1 ) ,where I c is the index set of the control variables.
We implement the simulation using the Summitpackage [8].
More details can be found in Appendix E.7.6.
We set a delay of t delay = 25 , andoptimize for T = 100 iterations.
The results of the experiment can be seen in Figure 3.
4.3 Finding Contamination Sources in Ypacarai Lake
[Figure] As a second real-world example that exhibits costly input changes, we investigate an alternative to thecase study introduced by [36].
Autonomous boats are used to monitor water quality in Ypacarai Lakein Paraguay.
We consider three different objectives, illustrated in Figure 5, created using the Schekelbenchmark function (as in [36]).
Each objective could correspond to a measure of water quality, suchas pH, turbidity, CO2 levels, or many more.
We consider the problem of finding the largest source ofcontamination in the lake by maximizing these objectives.
The cost between queries relates to theboat physically moving the boat from one place to another.
We assume that the boats can travel 10units, which corresponds to approximately 100km (due to fuel limitations), and that observations areavailable immediately.
We run optimization runs on each objective and shows the results in Table 1.
Conclusion and Discussion
This paper introduces and proposes a solution to the problem of optimizing black-box functionsunder costs to changes in inputs.
We have empirically shown that the regret achieved by our methodis comparable to those of classical Bayesian Optimization methods and we succeed at achievingconsiderably lower input costs, while being particularly well suited to asynchronous problems.
This setting, with input costs penalizing experimental changes, makes a major step towards automatingnew reaction chemistry discovery, e.g. in line with the vision of Lazzari et al.
We substantiallydecrease experimental cost with respect to classical black-box optimization, e.g. as used by Fathet al.
[7] and McMullen and Jensen [30].
In the real-life SnAR benchmark, SnAKe spends 45-55%of the cost while making similarly strong predictions to classical BO.
The synthetic benchmarks and
Ypacarai Lake example offer similar advantages.
Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope?
[Yes] We believe the claims in the abstract are backed by thepaper’s content.
(b) Did you describe the limitations of your work?
[Yes] See the last paragraph of Section4’s introduction.(c)
Did you discuss any potential negative societal impacts of your work?
[No] We do notbelieve there are any obvious negative societal impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms tothem?
[Yes]
If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results?
[N/A] See below.
(b) Did you include complete proofs of all theoretical results?
[No] While we include
some mathematical analysis in Section 3.6, it is not fully rigorous and it is only meantto illustrate the importance of the Point Deletion heuristic.3.
If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main experimental results (either in the supplemental material or as a URL)?
[Yes] The code isincluded in supplemental material, and will be made public after peer-review.
We alsogive details about how the random seeds for each run were chosen in the appendix.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how theywere chosen)?
[Yes] See Section E.
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)?
[Yes] See Section 4.
(d) Did you include the total amount of compute and the type of resources used (e.g., typeof GPUs, internal cluster, or cloud provider)?
[Yes] See Section 3.
If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators?
[Yes] See Section E.
(b) Did you mention the license of the assets?
[No] The licenses can be found in thedocumentation of each asset.
(c) Did you include any new assets either in the supplemental material or as a URL?
[No]But the GitHub repository URL to generate the paper’s result will be included after
peer-review.
(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating?
[N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content?
[N/A]
If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable?
[N/A]
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable?
[N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation?
[N/A]
A Motivating Example
Figure 6 shows a graphic of a droplet microfluidic reactor [44] which is our main motivating example.
[Figure]
N
Input Chemical Composition
Microreactor conditions, e.g. Droplet flow rate, Light exposure, Heat exposure
SYSTEM INPUTS
SYSTEM OUTPUTS
Online outputs, e.g. Fluorescence,
Confocal microscopy, Infrared camera
Offline outputs, e.g.
Chromatographic assays
[Figure]
B General Approach
While we present SnAKe as a detailed algorithm, the ideas behind it are far more general.
Forexample, in Section C we create our batch by taking samples from three different GPs.
Algorithm 3presents a more general algorithm that could be used for any similar task.
Algorithm 3 General Ordering-Based Optimization input : Optimization budget of T samples.
Method for creating batch of queries.
Method for creating an ordering from a batch of queries.
Method for updating paths.
begin : Create initial batch of size T and ordering, S for t = 1 , 2 , 3 , ..., T do if there is new information then
Update surrogate model
Choose a batch of new points to query
Create a new path, ˜ S
S ← ˜ S end if
Choose next query point from ordering: x t ← S t
Evaluate f ( x t ) end for
C Ypacarai Lake Experiment
A second motivating example for our research comes from optimization requiring spatially continuousexploration.
In this section, we examine the problem more closely, and show that SnAKe can easilybe extended to optimize multiple black-box functions simultaneously in the same search space.
The problem in question is an adaptation of the case study introduced by Samaniego et al.
Theapplication is to use autonomous boats to find the largest source of contamination in Lake Ypacarai in
Paraguay.
Figure 7 shows a visualization of the Lake, and three objectives we want to optimize over.
Each objective corresponds to a different measure of water contamination, examples might includepH, turbidity, CO2 levels, and more.
In Section 4.3 we explored optimizing over these examplesindividually.
[Table]
In this section, we extend the problem to optimize over all of these simultaneously .
This is importantbecause it is inefficient to run multiple objective runs when we are exploring the same search space.
At every iteration we choose one query x t , and obtain three outputs f 1 ( x t ) , f 2 ( x t ) , and f 3 ( x t ) onefor each objective.
We model each objective using an independent GP, meaning we can create Thompson Samples fromany objective .
Therefore we extend SnAKe by creating the batch using a mixture of samples fromeach objective.
For the experiment we use the ratio 1:1:1.
For TrEI and EIpu, we adapt to themulti-objective case by optimizing the first objective until we have travelled 3.3 units (approximately33km), after which we change to the subsequent objective.
Table 2 shows the results.
D Empirical Analysis of Escape Probability
[Figure]
D.1 Areas with stationary points
[Figure]
Figure 8 estimates the non-escape probability (see Definition 3.2) from the interval A = [0 . 1 , 0 . 2] .
The optimization objective is a bi-modal function.
Once we have 15 samples in the interval [0 . 1 , 0 . 2] ,we estimate the escape probability to converge to p ≈ 0 .
D.2 Areas without stationary points
We now repeat the same experiment as in section D.1, this time we change the interval to A =[0 . 0 , 0 . 1] which does not contain any stationary points.
One can observe a clear difference in thebehavior of p t as we include more information.
This time, p t → 0 very fast.
D.3 Resampling vs Point Deletion
Figure 10 empirically confirms the analysis of Section 3.7 showing the effectiveness of ϵ -Point
Deletion, and displaying the effect of increasing the budget from 100 to 250 iterations.
[Figure] [Figure]
[Figure] [Figure]
E Implementation Details
This section outlines the implementation details and hyper-parameter choices for all the methodscompared in the paper.
The code will be made available open source after peer review.
E.1 Computational Considerations (Extended)
Unfortunately, Algorithm 2 (SnAKe) is computationally expensive in two aspects.
First, for largebudgets, we may struggle to train and sample the GPs.
For our experiments, training was not an issueand we were able to use full model GPs.
However, we could use Sparse GPs [38] if needed.
The
Wilson et al.
[47] approach allows us to create GP samples efficiently, and possibly optimize themusing gradient methods.
The sampling can be done in linear time ( after the GP has been trained).
Weuse the Wilson et al.
[47] method to create our samples, and then optimize the samples using Adam[21].
[Figure]
r a t i
[Figure]
[Figure] [Figure]
a
[Figure]
i
[Figure]
I t
[Figure]
r a t i o n
[Figure]
Figure
: We investigate effect of Point Deletion, and empirically confirm our analysis from
Section
For Point Deletion, we calculate the escape prediction as pT p we estimated in Section D can see without Point
, the escape happens until until
Remark
Point see predictions actual
The second bottleneck is solving the TSP, which is NP-hard, and we may need to solve it almostat every iteration (for small values of t delay ).
There are heuristic solutions that give approximatesolutions quickly.
We use Simulated Annealing [22] which grows linearly with the budget size, T .
For small budgets, simulated annealing should find good solutions, but it could struggle as the budgetsize increases.
To solve this, note that we do not actually require a super-specific solution to theproblem: we are only expecting to query the first few points on a path before replacing it by anentirely new path.
We build an adaptive grid (at each iteration) consisting of two separate parts.
A very coarse grid,ξ global , covers most of the search space, and a very fine grid, ξ local,t , consisting of the N l samplesclosest to x t .
This allows us to define the grid ξ t = ξ global ∪ ξ local,t .
The remaining T − N l sampleswill be assigned to the closest point in ξ t (using the Euclidean distance).
The adaptive grid means we expect to have multiple samples assigned to the same point, specificallyin the coarse areas of our grid.
But this is not important, because we expect our immediate attentionwill be in the area around our current input where there should be little to no repetition.
This willallow the algorithm to focus on testing solutions which are relevant to our problem.
The adaptive grid introduces two hyper-parameters: the size of the global and local grids, respectively
N g and N l .
Using this method, we run the TSP heuristics on
E.2 Gaussian Processes
For every GP, we use the RBF Kernel with an output-scale, θ 0 :
[Equation]
where Θ = diag ( ℓ 1 , ..., ℓ d ) and ℓ i denotes the length-scale of the i th variable.
For the prior mean,we used a constant function with trainable value, µ 0 .
We implemented them all using the package
GPyTorch [9].
E.3 Training the hyper-parameters of the Gaussian Processes
Our method is well suited for physical systems.
Hence, we assume that there is good prior knowledgeof the hyper-parameters.
In particular, we found it reasonable that each hyper-parameter wouldbe given a lower and upper bound.
Normally, we would simply have a large initialization sample.
However, we believe this goes against the nature of the problem because we want to explore the spaceslowly to avoid large input costs.
So any type of initialization would be costly.
We simulate this in the following way: we first randomly sample max( T/ 5 , 10 d ) points from thed -dimensional input space, and train a GP on these data-points.
The hyper-parameters of the GP areoptimized by maximizing the marginal log-likelihood [35] over 500 epochs using Adam [21] with alearning rate of 0.01.
The resulting hyper-parameters will correspond to the ‘educated guess’.
Wethen set the following bounds:
a) For the length-scale, the lower bound is half the educated guess, and the upper bound doublethe ‘educated guess’.b)
For the output-scale, θ 0 , the lower bound is half the educated guess, and the upper bounddouble the ‘educated guess’.
c) For the initial mean, µ 0 , the lower bound is the educated guess minus a third of the initialvariance, the upper bound is the educated guess plus a third of the initial variance.
d) The noise parameter we simply set to be greater than 10 − 5 .
We (partially) enforce the constraints by setting a SmoothedBoxPrior on each parameter, with avariance of 0.001.
Finally, under the constraints defined above, we re-estimate the hyper-parametersevery time we obtain 25 new observations.
To make sure all models receive fair initializations, we set the same seed for each run and functionpair.
E.4 SnAKe
We used Simulated Annealing [22] to solve the Travelling Salesman Problem.
We implemented itusing the NetworkX package [14].
We initalized the cycle with the ‘greedy’ sub-algorithm and usedall default options.
We generated the Thompson Samples using the method introduced in Wilson et al.
[47] which weimplemented ourselves.
To optimize the samples we used Adam [21] and PyTorch [32] over 10 depochs, with a learning rate of 0.01.
We used 10 d multi-starts for each sample.
To create the samples,we used ℓ = 1024 Fourier bases.
For ℓ -SnAKe, we define an adaptive deletion constant, ϵ t = min( ℓ 1 ,t , ..., ℓ d,t ) , where ℓ i,t denotes thelength scale of the i th variable at time t (recall we are re-training the hyper-parameters every new 25observations, so the length scales change with time).
For the adaptive grid, we use N l = 25 local samples, and a corse global Sobol grid [40] of N g = 100points.
E.5 Classical Bayesian Optimization
We used BoTorch [3] to implement all methods in this section.
We optimized the acquisition functionsacross 150 epochs using Adam [21] with a learning rate of 0.0001 using 7500 random multi-starts.
E.5.1 Expected Improvement
Expected Improvement [31] optimizes the acquisition function:
[Equation]
where y best is our best observation so far.
Expected Improvement per Unit Cost [39] optimizes the acquisition function:
[Equation]
We set γ = 1 in all experiments to address the fact that not moving incurs zero cost.
Note that asγ → ∞ , the method will behave closer to normal Expected Improvement, and as γ → 0 the methodwill tend to stay closer and closer to the current input.
Truncated Expected Improvement [36] first maximizes the normal Expected Improvement acquisitionfunction.
It then travels a distance of at most ℓ towards the proposed point, where ℓ is the GPslength-scale.
We exclude this method from synchronous SnAr, as it is not suited for general costfunctions.
E.5.2 Upper Confidence Bound
Upper Confidence Bound [41] optimizes the acquisition function:
[Equation]
We set β t = 0 .
2 d log(2 t ) following Kandasamy et al.
E.5.3 Probability of Improvement
Probability of Improvement [23] optimizes the acquisition function:
[Equation]
E.5.4 Truncated Expected Improvement
Truncated Expected Improvement was developed by [36] trying to solve the task of using an automatedboat to monitor the water quality of Ypacari Lake in Paraguay.
The method seeks to take distancetravelled into account when doing Bayesian Optimization.
It does by selecting a point using Expected
Improvement, setting a direct path from our current input to the new point, and then truncating thepath one length-scale away.
That is:
[Equation]
E.6 Asynchronous Bayesian OptimizationE.6.1 Local Penalization
We use the penalization method as is described in González et al.
For the Lipschitz constant, weestimate it by calculating the gradient of µ t (using auto-differentiation) in a Sobol grid [40] of 50 dpoints and selecting L to be the maximum gradient in the grid.
UCB with Local Penalization We set β t = 0 .
2 d log(2 t ) and M = y best .
EIpu with Local Penalization We set γ = 1 .
E.6.2 Thompson Sampling
We use the sample procedure as in E.4, except we only optimize a single sample at every iteration.
E.7 Description of Benchmark Functions
We chose the benchmark functions to observe the behavior of SnAKe in a variety of scenarios.
Moredetails of all the benchmark functions can be found in Surjanovic and Bingham [42].
E.7.1 Branin2D
The two-dimensional Branin function is given by.
The function has three global maximums:
[Equation]
where we optimize over X
= [ − 5 , 10] ×
We set a = − 1 , = 5 .
1 / (4 π 2 ) , c = 5 /π , r = 6 ,s = − 10 , and t = 1 / (8 π ) .
E.7.2 Ackley4D
The four-dimensional Ackley function has a lot of local optimums, with the optima in the center ofthe search space.
The function is given by:
[Equation]
where we slightly shift the search space and optimize over X =
[ − 1 . 8 , 2 . 2] 4 .
This is to avoid havingthe optimum exactly at a point in the Sobol grid and giving SnAKe an unfair advantage.
We seta = 20 , = 0 . 2 , and c = 2 π .
E.7.3 Michaelwicz2D
The two-dimensional Michalewicz function is characterized by multiple local maxima and a lot offlat regions.
The function is given by:
[Equation]
where we set m = 10 and we optimize on the region
X
=
[0 , π ] 2 .
E.7.4 Hartmann
We select this function to see how the algorithms behave in similar functions as dimension increases.
We do three versions of the Hartmann function, with dimensions d = 3 , 4 , and 6 .
The equation isgiven by:
[Equation]

 where α = (1 , 1 . 2 , 3 , 3 . 2) T .
For d = 3 we use:
A =  
[Equation]
[Equation]
[Equation]
10 
[Equation]   3689 11704699 43871091 8732 5743  for d = 4 and d = 6 we use:
A =     3 .
1 
P = 10 −  1312 1696 82832329 4135 8307 3736 10042348 1451 3522 2883 30474047 8828 8732 5743 

They are all evaluated on the unit cube [0 , 1] d .
E.7.5 Perm10D
We select the 10-dimensional version of the Perm benchmark to test the capabilities of the algorithmsin a very high-dimensional setting (by BO standards).
The equation is given by:
[Equation]

[Equation]
where we set β = 10 .
We evaluate it on X =
[ − 10 , 10] d .
E.7.6 SnAr4D
We implement the simulation using Summit [8].
We control temperature between 40 and 120 degrees,concentration from 0.1 to 0.5 moles per liter, and residence time between 0.5 and 2 minutes.
Weset the cost parameters for temperature α temp = 5 , β temp = 1 , γ temp = 1 , for concentrationα conc = 2 , β conc = 0 .
01 , γ conc = 1 , and for residence time α residence = 3 , β residence = 0 .
05 andγ residence = 1 .
We further optimize over the equivalents of pyrrolidine between 1 and 5 units, butwe assume changing it incurs no input costs.
Since the SnAr benchmark is a multi-objective problem, we optimize a weighted sum of the twoobjectives:
[Equation]
where we set ω 1 = 10 − 4 and ω 2 = 0 .
We optimize over X =
[40 , 120] × [0 . 1 , 0 . 5] × [0 . 5 , 2] × [1 , 5] .
E.8 Ypacarai Implementation Details
As per [36] we model the Lake using the Shekel function which is given as: [Equation]
[Equation]
For the objective we use m = 2 , m = 3 , and m = 2 .
The other parameters are:
[Equation]
Where C ( i ) and β ( i ) represent the parameters of the i th objective.
All objectives are optimized acrosson a subset of X =
The subset is defined by creating grid of points mapping a high-resolutionblack and white image of Lake Ypacarai onto X .
For simplicity, we assume the cost of moving fromone point to another is simply the distance between the points, even though in practice we might needto take a longer route to avoid land.
For Truncated Expected Improvement, we simply project thetruncation into the closest point grid, to avoid sampling points outside of the Lake.
F Full Experiment Results
F.1 Tables of ResultsF.1.1 Synchronous Experiments
This section includes the full tables of results of the synthetic synchronous experiments.
The resultsare shown in Table 3 and 4.
F.1.2 Asynchronous Experiments
In this section we include the full table results of all asynchronous experiments.
The results areshown in Table 5 and 6. [Table] [Table]
F.1.3 SnAr Benchmark
We ran additional experiments on the SnAr benchmark.
For the first one (which includes the examplelooked at in the main paper) we tested on a budget of T = 100 iterations for different values of t delay .
The results are included in Tables 7 and 8.
We also carried out synchronous results on the benchmark, for different budgets.
The results areincluded in Tables 9 and 10.
F.2 Graphs for results of Section 4.
We include the full graphs of the sequential Bayesian Optimization experiments.
Each row representsa different budget.
The left column shows the evolution of regret against the cost used, the middlecolumn shows the evolution of regret with iterations, and the right column shows the evolution ofthe 2-norm cost.
The results encompass Figures 11 to 16.
The caption in each figure tells us thebenchmark function being evaluated.
Each experiment is the mean ± half the standard deviation of10 different runs.
[Table] [Table] F.3 Graphs for results of Section {+4.
+}
We include the full graphs of the asynchronous Bayesian Optimization experiments.
Each rowrepresents a different budget.
The left column shows the evolution of regret against the cost used,the middle column shows the evolution of regret with iterations, and the right columns show theevolution of the 2-norm cost.
The results encompass Figures 17 to 28.
The caption in each figuretells us the benchmark function being evaluated, and the time-delay for getting observations back.
Each experiment is the mean ± half the standard deviation of 10 different runs.
F.4 Graphs for results of SnAr Benchmark (section 4.2)
Figure 29 includes the whole set of results of the SnAr benchmark in the asynchronous setting.
Figure30 includes the results for the synchronous setting.
Each experiment is the mean ± half the standarddeviation of 10 different runs.
[Table]
and the best one in italics.
SnAKe consistently achieves regret comparable with BO methods.
EIpuLP performs well for small delays, but poorly as the delay is increased.
[Table]
[Figure]
[Table] [Table] [Figure] [Figure]
R
[Figure]
g r e t )
[Figure]
the budget, SnAKe outperforms two BO methods in regret, and outperforms all methods in cost.
ϵ = 0 gives the smallest cost of all at the expense of some regret.
[Figure] [Figure] R [Figure]
g r
[Figure]
t )
[Figure]
r
[Figure]
t )
[Figure]
g r e t ) [Figure] l
[Figure]
( R e g r e t )
[Figure]
the evolution of the 2-norm cost.
SnAKe outperforms has regret comparable with other methods for all budgets (UCB outperforms the rest for larger ones). SnAKe achieves significantly less cost at all budgets, this may be due to SnAKe exploring the many local optimums carefully. The first column shows that SnAKe achieves by far the best regret for low cost. In this example, SnAKe and EIpu have similar performance.
[Figure]
[Figure] e [Figure] [Figure]
[Figure]
R
[Figure]
g r [Figure] t )
[Figure] [Figure]
g r
[Figure]
t )
[Figure]
( R e g r e t )
[Figure]
did not update the model, we would achieve a much better performance (as it would be equivalent to Random).
We observe this behavior in the asynchronous case, where having a time-delay helps the method perform better (see asynchronous
Ackley, Figure 20).
EIpu and TrEI perform well in this example, we conjecture this is because they are doing far more localized searches while SnAKe is trying to cover all the space available (which is very difficult in higher dimensions).
[Figure]
[Figure] [Figure]
[Figure] [Figure]
R
[Figure]
g r e t )
[Figure] [Figure]
l
[Figure]
g ( R e g r e t )
[Figure]
Similar results to
, see
[Figure]
[Figure]
Similar results to the case with smaller delay, see Figure 23.
[Figure]
[Figure]
Similar results to other Hartmann benchmarks, see Figure 23.
[Figure] [Figure]
Similar results to other Hartmann benchmarks, see Figure 23.
[Figure]
R
[Figure]
[Figure]
l o g
[Figure]
R e g r e t )
[Figure]
l o g ( R e g r e t )
[Figure]
l o g ( R e g r e t )
[Figure] [Figure]