BO-Muse: A Human expert and AI teaming framework for accelerated experimental design 
Authors: Sunil Gupta
Alistair Shilton
Shannon Ryan
Arun Kumar Anjanapura Venkatesh
Majid Abdolshah
Hung Le
Santu Rana
Julian Berk
Mahad Rashid
Svetha Venkatesh
Venue: Submitted to ICLR 2023 (2023)
Keywords: Experimental Design
Machine learning
Optimisation
Bayesian optimisation
Human-AI Teaming

Abstract: In this paper we introduce BO-Muse, a new approach to human-AI teaming for the optimisation of expensive blackbox functions. Inspired by the intrinsic difficulty of extracting expert knowledge and distilling it back into AI models and by observations of human behaviour in real-world experimental design, our algorithm lets the human expert take the lead in the experimental process. The human expert can use their domain expertise to its full potential, while the AI plays the role of a muse, injecting novelty and searching for areas of weakness to break the human out of over-exploitation induced by cognitive entrenchment. With mild assumptions, we show that our algorithm converges sub-linearly, at a rate faster than the AI or human alone. We validate our algorithm using synthetic data and with human experts performing real-world experiments.

Under review as a conference paper at ICLR 2023
BO-MUSE: A HUMAN EXPERT AND AI TEAMING FRAMEWORK FOR ACCELERATED EXPERIMENTAL DE- SIGN
Anonymous authors Paper under double-blee blind review
ABSTRACT
In this paper introduce BO-Muse, a new approach to human-Al teaming for the optimisation of expensive blackbox functions. Inspired by the intrinsic dif- ficulty of extracting expert knowledge and distilling it back into Al models and by observations human behaviour in real- experimental design, our algo- rithm lets the human expert take the lead in the experimental process. The human expert can use their domain expertise to its full potential, while the Al plays the role of a muse, injecting novelty and searching for areas of weakness break the human out of expllxxplittttion induced by cognitive entrenchment. With mild as- sumptions, we show that our algorithm converges sub-linearly, at rate faster than the AI or human alone. We validate our algorithm using synthetic data and with human experts performing real-world experiments.
INTRODUCTION
Bayesian Optimisation (BO) (Shahriari et al. 2013) is popular sample-efficient optimisation tech- nique solve problems where the objective expensive. It has been applied successfully in diverse areas (Greenhill et al., 2020) including material discovery (Li et al. 2017), alloy design (Barnett al. 2020) and molecular design (Gómez-Booe Bombarelli al. 2018). However, standard BO typi- cally operates tabula rasa, building its model of the objective from minimal priors that do not include domain-specific detail. While there has been some progress made incorporating domain-specific knowledge to accelerate BO (Li et al.) 20188] Hvarfner et al. 2022) or transfer learnings from pre- vious experiments (Shilton et al.) 2017). remains the case that there significant corpus of knowledge and expertise that could potentially accelerate BO even further but which remain largely untapped due the inherent complexities involved in knowledge extraction and exploitation. In particular, this often arises from the fact that experts tend to organise their knowledge in complex schema containing concepts, attributes and relationships (Rousseau) 200T), making the elicitation of relevant expert knowledge, both quantitative and qualitative, difficult task.
Experimental design underpins the discovery of new materials, processes and products. How- ever, experiments are costly, the target func- tion is unknown and the search space unclear. To be sample-efficient, the least number of ex- periments must be performed. Traditionally experimental design is guided by (human) ex- perts who uses their domain expertise and intu- ition to formulate an experimental design, test it, and iterate based on observations. Living be- ings from fungi (Watkinson et al. (2055) to ants
Uptanic
Expert
B0-MMasc
Suggettion
Run experiments. for both suggestions
Suggettion
Figure 1: Bo-Muse Workflow.
(Pratt& & Sumpter 2006) and humans (Daw et al.) 2006; Cohen et al. they make these decisions: exploit the information they have, explore to gather new informa- tion. How humans balance this dilemma has been studied in Daw et al. (2006) examining human choices in n-arm bandit problem, they showed that humans were highly skewed towards exploita- tion. Moreover, when the task requires specialised experts, cognitive entrenchment is heightened and the balance between expertise and flexibility swings further towards remaining in known paradigms.
2007 face dilemma when

---

Under review as a conference paper at ICLR 2023
To break out of this, dynamic environments of engagement are needed to force experts incorpo- rate new points of view (Dane 2010). For such lateral thinking to catalyse creativity, Beaney/ (2005) has further confirmed that random stimuli are crucial. For example, using random stimuli to boost creativity has been attempted in the context of games (Yannakakis et al. 2014). In Sentient Sketch- book, machine creates sketches that the human can refine, and sketches are readily created through machine learning models trained on ample data. Other approaches use machine representations to learn models of human knowledge, narrowing down options for the human to consider. Recently, Vasylenko et al. (202T) constructs a variational auto-encoder from underlying patterns of chemistry based on crrppturecommposttion and then a human generated hypothesis guides possible solutions. An entirely different approach refines a target function by allowing machine learning to discover relations between mathematical objects, and guides humans to make new conjectures (Davies al.) 202T]. Note, however, that there is still a requirement for large datasets to formulate representations of mathematical objects, which is antithetical to samplc-efficincy as typically in experimental de- sign have budget on the number of experiments, data from past designs is lean, and formulation of hypothesis is difficult in this lean data space.
The of BO for experimental design overcomes the problems of over-exploitation and cognitive entrenchment and provides mathematically rigorous guarantees convergence to the optimal de- sign. However, as noted previously, this often means that domain-specific knowledge and expertise is lost. In this paper, motivated by our observations, rather than attempting to enrich Al models using expert knowledge expert take the lead in experimental design with the aid of an AI "muse" whose job is to augment the expert's intuition through AI suggestions. Thus the Al's role is provide dynamism to break an expert's s cognitive entrenchment and go beyond the state-of-the-ar in new problems, while the expert's 's role is harness their vast knowledge and extensive experience to produce state-of-the-ar designs. Combining these roles in formal framework is the main contribution of this paper.
accelcrate BO, we propose the BO-Muse algorithm that lets the human
BO-Muse is formal framework that inserts BO into the expert's workflow (see Figure 1), allowing adjustment of the AI exploit/explore strategy in response to the human expert suggestions. This process results in a batch of suggestions from the human expert and the Al at each iteration. This batch of designs is experimentally evaluated and shared with the human and the AI. The Al model is of BO-
updated and the process iterated until the target is reached. We analyse the Muse and provide sub-linear regret bound. We validate BO-Muse using optimisation benchmarks and teaming with experts to perform complex real-world tasks. Our contributions are:
Design of a framework (BO-Muse) for a human expert and an AI to work in concert to accelerate experimental design, taking advantage of the human's 's deeper insight into the problem and the Al's advantage in using rigorous models to complement the expert to achieve sample-efficiency;
Design of an algorithm that compensates for the human tendency to be overly exploitative by appropriately boosting the Al exploration;
Provide sub-linear regret bound for BO-Muse to demonstrate the accelerated convergence due to the human-AI teaming in the optimisation process; and
Provide experimental validation both using optimisation benchmark functions and with human experts to perform complex real-world tasks.
BACKGROUND
2.1 HUMAN MACHINE PARTNERSHIPS
Mixed initiative creative interfaces propose tight coupling of human and machine to foster creativ- ity. Thus far, however, research has been largely restricted to game design (Deterding et al.) 2017). where the authors identified open challenges including "what kinds of human-Al co-creativity can we envision across and beyond creative practice?"' Our work is the first example of the use such a paradigm to accelerate experimental design. Also of importance, though beyond the scope of this study, is the design of interfaces for such systems (Rezwana & Maher, (022) and how the differing ways human and machine express confidence affects performance (Steyvers et al. 2022
2

---

Under review as a conference paper at ICLR 2023
2.2 BAYESIAN OPTIMISATION
Bayesian Optimisation (BO, (Brochu et al. 2010 is an optimisation method for solving the prob- lems of the form: x argmaxxex [xx the least possible number of iterations when I is an expensive blackbox function. Bayesian optimisation models f* as a draw from a Gaussian Process (O K) with prior covariance (kernel) K [Rasmussen] 2006) and, at each iteration t, recommends the next function evaluation point by optimising a (cheap) acquisition function X + IR based on the posterior mean and variance given a dataset of observations ID,1
{(x, Ui) : E N,-1):
P--1 (x) y (K,- + d²1) 1 (x)
of_1 (x) = 1 (x,x) - KT_ (x) (Ki-1 + + - ki- (x)
[K(x,, x k))ls,jeN, and (x) [K (x,, x)] Experiments evaluate y/+ = (x,) + V4, where Vt is noise, the GP model is updated include the new observation, and the process repeats either until a convergence criteria is met or fixed budget of evaluations is exhausted. Common acquisition functions include Expected Improvement (HI, (Jones et al. (998)) and GP-UCB (Srinivas et al. 2012). GP-UCB uses:
where Yt-1 [VIlEEN. is the set of observed outputs, K
at (x) = P (x) p 31/20--100 01-1 (x)
(I)
Here 3e is a variable controlling the trade-off between exploitation of known minima (if 3, small) and exploration (if 3, is large). The optimisation performance depends on B.. To achieve sub-linear convergence Choowdhury & Gopalan (2017) recommend 3, = Xt. where:
Σ 2 (3) + 1 + + ||
(2)
X =
and % is the maximum information gain (maximum mutual information between f and any t observations (Srinivas et al., (0012). Typically it is assumed that f ~ GP K) or f* E Hk. where Hx is the reproducing kernel Hilbert space of the kernel K (Srinivas et al. 2012). Otherwise the problem is mis-specified. One approach to mis-specified BO enlarged confidence GP-UCB (EC-GP-UCB, (Bogunovic & Krause 202I)). In EC-GP-UCB, the function closest to the objective Too. and the acquisition function takes the form:
f is denoted f argmin
ag (x) Ph- 1 (x) + (8)/²+ Evt O--1 (x)
where c =
- 10 is the mis- mpessiecciiiation gap. The additional term in the acquisition function required to ensure sub-linear convergence in this case.
-
FRAMEWORK
We consider the following optimisation problem x* argmax f* (x). where f : R expensive and evaluation is noisy, with results y = (x) +u, where v is E-sub-Gaussian noise. We optimise I" as series of experimental batches, indexed by 1, 2, ..... For each batch a human expert suggests x, for evaluation and an AI suggests X, (hatted variables relate to the human expert, breved variables to the AD). Both experiments are carried out, the human expert and the Al update their models based on the results, and the process is repeated for a total of S batches.
We do not know precisely how the human expert will model f*, but the Al maintains GP model with kernel (prior covariance) K, and we assume the human expert does the same with some un- known kernel K2 (we discuss this in detail in section 3.1, where K2 and K2 may be updated after each batch. The posterior means and variances given dataset D.- = {(8,, Di), (Xi, 2i) ; S 8) are:
Pe (x) = y (K, + - (x),
(x) 82 (x) K, (x,x) - KT (x) (K, + 1 (x)
62 (x) = R, (x, x) - i. (x) (K, + 0²
Hg (x) = y. [K, + -
for the human and AI, respectively. We assume f E H. = HRg lies in the RKHS of the Al's kernel K., We do not make this assumption for the human expert, making the problem mis-specified from their perspective. Hence borrowing from (Bogunovic & Krause 2021I. assume that for batch the human by default attempts maximise the closest function to f" in H.:
f = srgmin SER.::1 <B IlS - Il
(here H. H k.)
3

---

Under review as a conference paper at ICLR 2023
We further assume that the gap between f* and f is bounded as 1 f 1lo0 Eg. and that the human is able to learn (in effect, update their kernel) to "close the gap", so lim 6, - 0.
The AI generates recommendations using GP-UCB. Motivated by (Borji & Iti, 2013). we assume the human in effect generates recommendations using EC-GP-UCB (Bogunovie & Krause 2027
x, = xxxmi (x) = P 1 (x) + (B./2 2 + & 22)66_116
(4)
x, argmin (x) 1 (x) 10-10,116 (x)
The form of â₂ captures the broad behaviour of human recommendation selection, balancing ex- ploitation of known good regions, which corresponds to 3, + 0 (e.g. "based on my model, this experiment should yield good results"), and exploration of areas of uncertainty, which corresponds to 3. + 0o (e.g. "explore here I'm curious"). We do not presume to know the precise trade-off 3, used by the human but, based on (Borji & Itti 2013 Dane, 2010). it appears probable that the human will pursue a conservative policy, so B, may be small. We therefore use the Al trade-off B3,. which we control, compensate for the potential conservative tendencies of the expert.
is convenient to specify the exploration/eeplootttoon trade-off parameters 3 and 3, relative 2 (Chowdhury & Gopalan 2017 [Bogunovic & Krause, 2021). Without loss of generality we require
that B3 E (SaiXe: SotxX and 13 = SoXo. where 0 I Sot S 1 Eot I co, Co 2 and 2
8. = - 2n (1/6) + + % 11 11 ) X. = ( 2 (166 + +1 + + llx) (5)
where % and %e are, respectively, the max information-gen -aaiioonnnnnnninn for human expert and the AI. We show in section (.3 that the this suffices to ensure sublinear convergence for any human expert selections, and moreover that the convergence rate can be improved beyond what can be achieved by standard GP-UCB if the human operates as described here.
3.1 THE HUMAN MODEL.
We posit that the human expert will maintain, explicitly or implicitly, an evolving weight-space model (we do not presume to know the details of this, only that it exists, explicitly or otherwise):
(x) = Is (p. (x))
of f", where 9. is in some sense "simple" and p. R" - T represents the human's 's understand- ing of important features. This fits into the above scheme we let the form of Å. dictate the kernel Kg. For example if we know that 9. is linear (so (x( = w (x) in weight-space) then we can say that the human is, in effect, using GP model with linear-derived model:
(x,x') Po (x) p. (x')
Similarly if the human is using a model Ji.. captured by a -order approximation model fx(x). = w (x]p(())))== then we can assume a GP with polynomial-derivel kernel:
K, (x, x') = (1 + P (x) p. (x'))
As discussed previously we do not assume that the human expert's evolving model suffices to com- pletely replicate ʃ*, particularly in the earlier stages of the algorithm, but we do assume that the human expert is capable of learning from observations of f* refine their model, closing the gap ε2. between their model and f* so that lim, ε2.- = 0.
With regard max information gain, we can reasonably assume that the human expert (the expertise is important) starts with a better understanding of f" than the AI - the underlying physics of the system, the behaviour one might expect in similar experiments, etc. So the human expert may begin with an incomplete but informative set of features that relate to their knowledge of the system or similar systems similar, or an understanding of the covariance structure of design space, so:
The prior variance of the human expert's 's GP will vary between eero-knowledge base level in regions that are a mystery to the expert, and much lower in regions where the expert has good understanding from past experience, understanding of underlying physics etc.
4

---

Under review as a conference paper at ICLR 2023
The prior covariance of the expert's GP (the expert's kernel) will have structure informed by the expert's 's understanding and knowledge, for example, of the "region A will behave like region B because they have fcature/atrribute C in common" type, an experiment in region A will reduce the human expert's ' posterior variance both in regions.
By comparison, the Al will typically start with a generic kernel prior like an SE kernel, Matern kernel or similar. Such priors are "flat" over the design space, with no areas of lower prior variance and no "region A will behave like B" behaviour, variance Thus as the algorithm progresses the human expert's 's variance will both start from a lower prior and decrease more quickly than the Al's. We know that max information gain is bounded the sum of the logs of the pre-experiment posterior variance (Srinivas et al., 2012 Lemma 5.3), so we may reasonably assume that the expert's 's max information gain will be lower than the Al's. Finally, we argue that, as the human expert's 's kernel is built on relatively few, highly informative features, the expert's RKHS norm [| ll.. |/ will be less than the Al's norm ||/" [/|2.
experiment will only reduce the Al's local
3.2 THE BO-MUSE ALGORITHM
The BO-Muse algorithm is shown in algorithm ! where f* is optimised using sequence of batches 1, , S, each containing one human and one Al recommendation, respectively &, and X,. The Al recommendation minimises the GP-UCB acquisition function on the Al's GP posterior, with the exploration/exploittion trade-off 33 given (see section 3.4). The human recommendation is assumed to be implicitly selected minimise the EC-GP-UCB acquisition function (3), where the
exploitation/exxporaiiin trade-off 3, is unknown but assumed to lie in the range B, E & &tt and the gap é is unknown but assume converge to 0 at the rate to be specified in theorem D
We two approximations in our definition of B_. Following the standard practice, approximate the max information gain %/2 as _x....)e In(1 + o 23, (x)); and we approximate the RKHS
norm bound B = ||, as || H., which is updates using max to ensure it is non-decreasing with s (this will tend over-estimate the norm, but this should not affect the algorithm's 's rate of convergence). For simplicity we assume that o = Σ (the model parameter matches the noise).
Algorithm 1 BO-Muse
Input: Initial observations Do = {x11, (X2, 92). ,....,, Al prior GP(0, K). Let B = 1. for E 1, 2, ., S do
Set 3. 7( V (m(1/)) + 1+ + Σ (x.... In (1 + σ 2 (x)) + B3) as per 5), 8
Al recommends X, as X = argmaxxexd, (x) = Wis- (x) + 2112 1 8 (x).
Human recommends X.. Run experiments to obtain g. = fo (xx) + ig and d. = f (x.) + E..
Set DD, I1. 1D,-1 U {(x,,, M.), (x,, }1}} and update the AI GP posterior.
Set B = max(B,x y (K, + OPI) yz}.
end for
3.3 CONVERGENCE AND REGRET BOUNDS
We now discuss the convergence properties of BO-Muse. Our goal here is twofold: first we show that BO-Muse converges even in the worst-case where the human operates arbitrarily; and second, assuming the human behaves according to assumptions, ated. Our approach is based on regret analysis (Srinivas et al.] 2012 Chowdhury & Gopalan [017] Bogunovic Krause 2021). As experiments arebatched we are concerned with instantaneous regret per batch, not per experiment. The instantaneous regret for batch is:
analyse how convergence is acceler-
Ta min {f" (x*) - f (K.) f (x*) (x,))
and the cumulative regret up to and including batch S is Rs = Ei If Rs grows sub-linearly then the minimum instantaneous regret will converge to as S xC.
We do not consider the possibility of transfer learning of this structure, as this requires the expert to distill their knowledge in an amenable form which, as noted previously, is highly non-trivial.
5

---

Under review as a conference paper at ICLR 2023
Consider first the worst-case scenario, i... arbitrary, non-expert human. In this case the BO- Muse algorithm I effectively involves the Al using a GP-UCB acquisition function (the constant scaling factor on B, does not change the convergence properties of GP-UCB) to design a sequence of experiments Xs. where each experiment costs twice as much as usual evaluate and yields two observations, (&a, Ws = (*x)) + ri.) and (&,99. = (*.) + fi,), where &, is arbitrary. Additional observations can only improve the accuracy of the posterior, so using standard methods (e.g. Chowwhury & Gopalan (2017); Bogunovic & Krause] (2021): [Srinivas et al. (2012), we see
that Rs = 125222 + v (In(1/8) + T2s) which is sub-linear for well-behaved kernels.
Thus we see that the worst-case convergence of BO-Muse is the same, in the big-O sense, as that of GP-UCB. However this is pessimistic: in reality we assume human is generating experiments x, using an EC-GP-UCB style trade-off between exploitation and exploration, and moreover that the human is an expert with an implicit or explicit evolving model of the system that is superior to the Al's generic prior. For this case we have the following result:
Theorem 1. Fix 8 > 0g > O, 0, E 10,, ool, S E N4. Assume Cي E (0, 11, 5. = 22 so: 21/2 2 ) Sel for all S S. where 0--1t max(ox-1(x*)) xx))xx))), 0--11
(ex(xxx(x (2 -1-
ex)xe (1
1
(6)
ox,))x (x"))
Mo(a, b) = (2 + +b9)) is the generalised mean, and Xa, Xs as per (5). if = 0(s 3) then:
JR S In ( 0, (exp (Voxgge
exp( (VC(((( (1/22)))) 2 P803s
3
(7)
with probability 2 1 5, where C, 0 are constants.
We present the proof of this theorem in the appendix. As discussed previously is and is are the max information gains for the human and AI, respectively; Čes (0, 1] defines the degree which the human is allowed to over-exploit; and Ç defines the degree to which the AI must over-explore to compensate. The parameters 02 constrain the range of Č for which the theorem is applicable through the condition (6). and the "mix" between human and AI in the regret bound 77). Assuming the max information gain for the human model has better convergence properties than the Al model, which we consider reasonable based on previous discussion, and noting that the power mean (o b) will be dominated by max(a, b) as 0 + co and by min(a, b) as 0 + -00, we see that:
When 0, is large the AI trade-off parameter Č must be large to satisfy (6). corresponding very explorative AI, and the regret bound (7) will be asymptotically superior, being dominated by the human's max information gain tss for large S which, as discussed in section 3.1 may be reasonably expected be smaller than the Al's max information gain due to the human expert's superior understanding of f*.
When 0, is small the AI is less constrained, so the theorem is applicable for more ex- poitative AIs. but the regret bound (7) will be more strongly influenced by the Al max information gain and hence asymptotically inferior.
Note that, while larger 0g will lead to tighter regret bounds in the asymptotic regime, this bound will only apply for very explorative AIs, and moreover the factor (1(1 + <1/2))² so the regret bound may only be superior for very large S. The "ideal" trade-off is unclear, but the important observation from this theorem is that the human's 's expertise, and subsequent superior maximum information gain, will improve the regret bound and thus convergence.
(1(1222))) will also be large,
3.4 TUNING PARAMETER SELECTION
We now consider the selection of the tuning parameter (9, to ensure faster convergence than standard
BO alone. The conditions that 3. must meet are specified by ( Equivalently, taking the maxi-
mally pessimistic view of human under-exploration (i.e. E. = 0), we require that exp(0, VX
21/2 ),1")..-11) 5 2 - exp(0.//x..--11) or, equivalently, & 2 (1 + (5z (32) 1-- g.-1t 1 In (-+ ))² where 6. = 8, B. E (0, In 2). and, noting that the right of the inequality strictly increas-
ing, Φ. = corresponds to č = 1 + 1/2 011 2--12 and Φ. = In 2 to č. = xc.
0--1
We need o. to ensure 0, and note that h(0) = 1 Φ In( increasing and h(In 2o.

---

Under review as a conference paper at ICLR 2023
Table 1: Synthetic benchmark funcctinns Analytical forms are provided in the second column and the last column depicts the high level features used by simulated human expert.
Functions
f (x)
High Level Features
x = x1.
Ma
0.26 * (of + .²) 0.48 * (0) * *22)
J/3 =
5 = cos(21). 2/2 = cos(
Ackley-4D 4D
bv 21|=113 eV VV cosfex;) a a
4/33 cos((3)) )
|||||2
sin " (Tw1) + + (wa - [1 + sin" sinnnnnnnni (2Twa))
5 # (sin 21)² )²
LD
where:
Σx- (wi 1) + 10 10 s + 1)]
2 (sin rry)
and wy = 1 + اليد Vi E No
Vj No
it reasonable to assume that the max information gains satisfy 7s 2 9ss k. With these assumptions VX. S VX.. Furthermore we prove in the and that || Il, S
As discussed in section
3.1
Appendix that lim3-+o0 55-15 1, so we approximate the bound on ć. as C. 2 (1+ &h In( () Recalling ds (In 1, In 2) finally, somewhat arbitrarily select &, In 3/2 (the middle of the range in the log domain), which leads to the following heuristic used in the BO-Muse algorithm:
& 2 (1+
1n In3/2
2 Rd 7
(8)
EXPERIMENTS
We validate the performance of our proposed BO-Muse algorithm in the optimisation of synthetic benchmark functions, and the real-world tasks involving human experts. In all our experiments, we have used Squared Exponential (SE) kernel at all levels with associated syper-parameters tuned using maximum-likelibee
estimation. We measure the sample-efficinncy of BO-Muse framework and other standard baselines in terms of the simple regret (ri): rz I [x(x maxx,eD, f'(xx), where fx"" ) is the true global optima and (xx;) is the best solution observed in t iterations. Experiments were run on an Intel Xeon CPU@ 3.60GHz workstation with 16 GB RAM capacity.
4.1 EXPERIMENTS WITH OPTIMISATION BENCHMARK FUNCTIONS
We have evaluated BO-Muse on synthetic test functions covering range of dimensions, as detailed in Table T We compare the sample-efficiens of BO-Muse with (i) Generic BO: A standard GP- UCB based BO algorithm with the exploration-exploitaiee
trade-off factor (3) set as per SSinivas) jet al. (2012): (ii) Simulated Human: A simulated human with access to higher level properties (refer to Section 3.1 and high level features in Table D that may help to model the optimisation function more accurately; and (iii) Simulated Human + PE: A simulated human teamed with an
AI agent using a pure exploration strategy (that is, an AI policy with & = oc). To simulate a human expert (with high exploitation),
standard BO algorithm with small exploration factor maximising â. (x) = (x) + 0.0016, (x). Furthermore, we have ensured to allocate the same function evaluation budget for all the competing methods.
Figure 2a and 2b shows the simple regret computed for the Ackley-4D and Levy-6D functions averaged over randomly initialised rums BO-Muse consistently outperforms all the baselines for both functions. We believe that the poor performance of Simulated Human + PE is due to the pure exploration strategy used. Additionally, we have conducted an ablation study by varying the degrees of exploitation-exploraaii strategy (3.) to simulate covering over-exploitative to over- explorative experts. Figure 2c shows the result of the ablation study for Matyas-2D function and we refer to the appendix for the additional results. To further understand the behaviour of BO-Muse framework with other human acquisition functions, optimisation experiment to include Expected Improvement acquisition function for human experts. The experimental details and results of the ablation study are provided in the Appendix.
have also considered additional function

---

Under review as a conference paper at ICLR 2023
AD function
LevyID
Study Malyas
A-1
22
Sinddtdd numon
Gore
23
Cunox
Seouiet framon
30
A-10
Simp Regget
Strege Sergre
A 1030
-2
24-
12
25 25
24
13
14
Functio Evaluations
Function Evaluadion
Function Exalusioon
(a)
(b)
(c)
Figure 2: Simple regret versus iterations for (a) Ackley-4D (b) Levy-6D functions. Ablation study with varying degrees of exploitation-exploration parameter (3,) is shown in (c).
4.2
REAL-WORLD EXPERIMENTS
We now evaluate the performance of our proposed framework in complex real-world tasks.
4.2.1 1
CLASSIFICATION TASKS SUPPORT VECTOR MACHINES AND RANDOM FORESTS
Experimental Set-up. The expert task is to choose hyper-parameters for Support Vector Machine (SVM) and Random Forest (RF) classifiers operating on real-world Biodeg dataset from UCI repos- itory (Dua & Graff, 2017). We divide the dataset into random 80/20 train/test splits. We set up two human expert teams. Each member of Team 1 works in partnership with BO-Muse, whilst mem- bers of Team 2 work individually without BO-Muse. We recruited 8 participants coneripsnnncccssstiinnnnn of 4 postdocs and 4 postgraduate students. 2 postdocs and 2 students are allocated to each team ran- domly so that each team has 4 participants, with roughly similar expertise. Each participant is given the same budget, 3 random initial designs + 30 further iterations. At the end of each iteration, the test classification error for the suggested hyper-parameter set is computed. We measure the overall performance using simple regret, which will be the minimum test classification error observed so far. The individual results from each team are averaged to compare (Team 1) BO-Muse + Human vs (Team 2) Human alone (baseline). Additionally, we also report the performance of Generic BO (AI alone) method to demonstrate the efficacy of our approach. Further, we have set the same seed initialisation and allocated the same evaluation budget for all the algorithms. As provide each participant the same set of random initial observations, we do not compute the descriptive statistics for the Generie BO baseline. We note that BO-Muse is designed work with experts at various levels as we assume an imperfect expert model via a mis-specified GP.
Interfacing with Experts. The human experts perform two hyper- hhper-paraman parameter tuning experiments to minimise the test classification error of SVM and RF. The expert is provided with simple graphi- cal interface (Figure 3a) which shows accumulated observations of classifier performance as a func- tion of paprapparaetter, with the best result so far shown in dark blue. Interfaces for both teams are similar excepting that teams working with BO-Muse also see the previous Al-generated obser- vations indicated by v. Experts suggest the next hyper-parameter set by clicking at a point of their choice inside the plot. We refer to Appendix A.9 for more details.
Experiment - SVM Classification. In this experiment we have considered C-SVM classifiers with Radial Basis Function (RBF) kernel. We have used LibSVM (Chang Lin 2011 imple- mentation of SVM with hyper-parameters i.e., y and C are tuned in the exponent space of [33.3].
kernel scale and the cost parameter C. The SVM
Experiment Random Forest Classification. In the classification tasks with random forests we let the experts and BO-Muse tune the hyper-parameter maximum depth of the decision tree and the number of samples per split in the range (0, 100), and (1,50) respectively.
Necessary ethics approval obtained.
8

---

Under review as a conference paper at ICLR 2023
Zrlar Suggession
6.6
Hypersaramett
Hygerparamete
Expert alone R0-Muse Humme
Expart alone
Test Clessilitaation Error
Text Clossiiication Erros
9.28
BO-Muse Humann
Generic BO
0.26
Gonoric BO
0-
013
0.23
-2
612
0.30
0.28
611-
-4
-2
2
20
26
23
10
15
25
Gamma
Function Evalusaions
Function Evalustions
(b)
(c)
best per- formances indicated in blue (darker is higher). If working with BO-Muse, machine point from the previous iteration (#) are shown with v: Simple regret vs iterations for the hyper-paramete, tuning of classifiers comparing Team 1 (BO-Muse + Human) (red) Team 2 (Human only) (black) vs Al alone (blue). We report the simple regret mean (along with its standard deviation) for (b) Support Vector Machines (SM and (c) Random Forests (RF) classifiers.
Figure 3: (a) Interface used by participants showing performance vs
The results of our classification experiments are depicted in Figure and and Figure 3c In both exper- iments, the BO-Muse + Human team outperforms the experts working on their own.
4.2.2 SUPPLEMENTARY SPACE SHIELD DESIGN EXPERIMENT
Our third experiment evaluates BO-Muse in real applied engineering experiment. We consider the design of shield for protecting spacecraft against the impact a space debris particle by partnering with a world leading impact expert. For this problem, i.e., impact of cubic steel debris particle, there exists no state-of-the-roe art solution. We include this experiment in Appendix AA because of the lack of familiarity of CS with this problem and the detail required give sufficient background.
In the experiment (see Table 6 in Appendix AA.10) we observe the human expert initially exploring solutions based on the -tate-o--the-art for more typical debris impact problems (which are normally simplified to spherical aluminium projectiles). The expert is observed to rapidly exploit their initial 3 designs to identify two feasible shielding solutions within the first four batch iterations (ID 4-11, marked in blue). The expert performs further exploitation of these successful designs (Result=0) over the next four batch iterations (ID 12-19, marked in brown) in attempt reduce the weight, but is unsuccessful. To this point the expert does not appear to have been influenced at all by the BO suggestions. However, the next four batch iterations (ID 20-27, marked green) we can observe the expert taking inspiration from the previous BO-Muse suggestions. One such exploitation results in successful solution (ID 27), further exploitation of which provides the best solution identified by the experiment ID 29. This solution is highly irregular for spacecraft debris shields, utilising polymer outer layer contact with a metallic backing to disrupt the debris particle. Such a design is not reminiscent of any established flight hardware, Muse is demonstrated to have performed its role as hypothesised, inspiring the humaa expert with novel designs that are subsequently subject exploitation by the human expert. The experimental set-up and results are discussed in detail in Appendix 1. 10)
e.g., Christiansen et al. (2009). Thus, the BO-
CONCLUSION
We have presented a new approach to human expert/Al teaming for experimental optimisation. Our algorithm lets the human expert take the lead in the experimental process thus allowing them to fully use their domain expertise, while the AI plays the role of a muse, injecting novelty and searching for regions the human may have overlooked to break the human out of over-exploitation induced by cognitive entrenchment. We show that our algorithm converges sub- aub-linearly and faster than either the AI or human expert alone.

---

Under review as a conference paper at ICLR 2023
REFERENCES
CE Anderson Jr, IS Chocron, and AE Nicholls. Damage modeling for taylor impact simulations. Journal of Applied Physics IV, 134:331-337, 2006.
Anon. Metallic materials and elements for aerospace vehicle structures. Technical Report MIL.- HDBK-5J, US Department of Defense, 2003.
Anon. Aisi 4340 steel, normalized, 100 mm in.) round. Technical report, Aerospace Specification Metals (ASM) Inc., 2022.
Biswajit Banerjee. The mechanical threshold stress model for various tempers of aisi 4340 steel. International Journal of Solids and Structures, 44:834-859, 2007.
MR Barnett, M Senadeera, D Fabijanic, KF Shamlaye, J Joseph, SR Kada, S Rana, S Gupta, and Venkatesh. A serap-tolerant alloying concept based high entropy alloys. Acta Materialia, 200:735-744, 2020.
Michael Beaney. Imagination and creativity, volume 4. Open University Worldwide Ltd, 2005.
Ilija Bogumovic and Andreas Krause. Misspecified gaussian process bandit optimization. volume 34, 2021.
Ali Borji and Laurent Itti. Bayesian optimization explains human active search. In Advances in neural information processing systems, volume 26, 2013.
Eric Brochu, Vlad M. Cora, and Nando de Freitas. A tutorial on bayesian optimization of expensive cost functions, with applications to active user modeling and heirarchical reinforcement learning. eprint arXiv: 0012 2.999, arXiv.org, December 2010.
Chih-Chung Chang and Chih-Jen Lin. Libsvm: a library for support vector machines. ACM trans- actions intelligent systems and technology (TIST), 2(3):1-27, 2011.
Sayak Ray Chowdhury and Aditya Gopalan. On kernelized multi-armed bandits. Doina Pre- cup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 844-853, International Convention Centre, Sydney, Australia, Aug 2017. PMLR.
Eric Christiansen, Jeanne Crews, J.E. Williamsen, J.H. Robinson, and A.M. Nolen. Enhanced me- teoroid and orbital debris shielding. International Journal of Impact Engineering, 17:217-228, 1995.
Eric Christiansen, Jim Arnold, Alan Davis, James Hyde, Dana Lear, J.-C. Liou, Frankel Lyons, Thomas Prior, Martin Ratliff, Shannon Ryan, Frank Giovane, Bob Corsaro, and George Studor. Handbook for designing mmod protection. Technical Report NASA/TM-2009-214755, NASA Johnson Space Center, 2009.
Jonathan D Cohen, Samuel M McClure, and Angela J Yu. Should i stay should go? how the human brain manages the trade-off between exploitation and exploration. Philosophical Trans- actions of the Royal Society B: Biological Sciences, 362(1481):933-942,2 2007.
Brooke Corbett. Numerical simulations of target hole diameters for hypervelocity impacts into elevated and room temperature bumpers. International Journal of Impact Engineering, 33:431- 440, 2006.
Burton Cour-Palais and Jeanne Crews. A multi-shock concept for spacecraft shielding. International Journal of Impact Engineering, 10: 10:135-146, 1990.
Erik Dane. Reconsidering the trade-off between expertise and flexibility: cognitive entrenchment perspective. Academy of Management Review, 35(4):579-603, 2010.
Alex Davies, Petar Veličković, Lars Buesing, Sam Blackwell, Daniel Zheng, Nenad Tomašev, Richard Tanbur,, Peter Battaglia, Charles Blundell, András Juhász, et al. Advancing mathematics by guiding human intuition with ai. Nature, 600(7887):70-74, 2021.
10

---

Under review as a conference paper at ICLR 2023
Nathaniel D Daw, John P O' O'doherty, Peter Dayan, Ben Seymour, and Raymond J Dolan. Cortical substrates for exploratory decisions in humans. Nature, 841 879, 2006.
Roberto Destefanis, Moreno Faraud, and Marco Trucchi. Columbus debris shielding experiments and ballistic limit curve. International Journal of Impact Engineering, 23:181-192, 1999.
Sebastian Deterding, Jonathan Hook, Rebecca Fiebrink, Marco Gillies, Jeremy Gow, Memo Ak- ten, Gillian Smith, Antonios Liapis, and Kate Compton. Mixed-initiative creative interfaces. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems, pp. 628-635, 2017.
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http::/archive ics uci mrci.eed//m
Christian Fiedler, Carsten W Scherer, and Sebastian Trimpe. Practical and rigorous uncertainty bounds for gaussian process regression. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 7439-7447, 2021.
Rafael Gómez-Bombarelll, Jennifer N Wei, David Duvenaud, José Miguel Hernández-Lobato, Benjamín Sánchez-Lengeling, Dennis Sheberla, Jorge equilera-Iparraguire Timothy D Hirzel, Ryan P Adams, and Alán Auzuru-Guzik. Automatic chemical design using data-driven contin- uous representation of molecules. ACS central science, 4(2):268 276, 2018.
S. Greenhill, S. Rana, S. Gupta, P. Vellanki, and S. Venkatesh. Bayesian optimization for adaptive experimental design: A review. IEEE Access, 8:13937-13948, 2020. doi: 10.1109/ACCES. 2020.2966228.
Eduard Gruneisen. The state of solid body. Technical Report R19542, National Aeronautics and Space Administration (NASA), 1959.
John Hallquist. Ls-dyna theory manual. Technical report, Livermore Software Technology Corpo- ration (LSTC), Livermore, 2006.
Teri Hamlin, Michael Canga, Roger Boyer, and Eric Thigpen. 2009 space shuttle probabilistic risk assessment overview. In 10th International Probabilistic Safety Assessment and Management Conference, 2013.
TJ Holmquist, DW Templeton, and KD Bishnoi. Constitutive modeling of aluminum nitride for large strain, high strain-rate, and high-pressure applications. International Journal of Impact Engineering, 25:211-2311 2001.
Carl Hvarfner, Danny Stoll, Artur Souza, Marius Lindauer, Frank Hutter, and Luigi Nardi. T-bo: Augmenting acquisition functions with user beliefs for bayesian optimization. arXiv preprint arXiv:2204.11051, 2022.
Gordon Johnson and William Cook. A constitutive model and data for metals subject to large strains, high strain rates and high temperatures. In 7th International Symposium on Ballistics, 1983.
Gordon Johnson and William Cook. Fracture characteristics of three metals subjected various strain, strain rates, temperatures and pressures. Engineering Fracture Mechanics, 21:31 1985.
Donald R. Jones, Matthias Schonlau, and William J. Welch. Efficient global optimization of expen- sive black-box functions. Journal of Global optimization, 13(4):455 492, 1998.
Paula Krisko. The new orbital debris engineering model ordem 3.0. In AIAAAAS Astrodynam- ics Specialist Conference, 2014.
Cheng Li, David Rubín de Celis Leal, Santu Rana, Sunil Gupta, Alessandra Sutti, Stewart Greenhill, Teo Slezak, Murray Height, and Svetha Venkatesh. Rapid bayesian optimisation for synthesis of short polymer fiber materials. Scientific reports, 7(1): 2017.
Cheng Li, Santu Rana, Sunil Gupta, Vu Nguyen, Svetha Venkatesh, Alessandra Sutti, David Rubin, Teo Slezak, Murray Height, Mazher Mohammed, and Ian Gibson. Accelerating experimental design by incorporating experimenter hunches. In Data Mining (ICDM), 2018 IEEE International Conference on, pp. 257-266. IEEF, 2018.
11

---

Under review as a conference paper at ICLR 2023
Long Nguyen, Torsten Lassig, Shannon Ryan, Werner Riedel, Adrian Mouritz, and Adrian Orifici. A methodology for hydrocode analysis ultra-high molecular weight polyethylene composite under ballistic impact. Composites: Part A, 84:224-235, 2016.
Josip E. Pečarić. Generalization of the power means and their inequalities. Journal of Mathematical Analysis and Applications, 161( 404, 1991.
Stephen C Pratt and David JT Sumpter. A tunable algorithm for collective decision-making. Pro- ceedings of the National Academy of Sciences, 103(43):159061133 2006.
C.W. Pyun. Generalized means: Properties and applications. American Journal of Physics, 42(10):
896-901, 1974.
Carl Edward Rasmussen. Gaussian processes for machine learning. MIT Press, 2006.
Jeba Rezwana and Mary Lou Maher. Designing creative ai partners with cofi: A framework for modeling interaction in human-ai co-creative systems. ACM Transactions on Computer-Humee Human Interaction, 2022.
M Rivero, B Shiotani, M Carrasquilla, N Fitz-Coy, J-C Liou, M Sorge, T Huynh, J Opiela, P Krisko, and H Cowardin. Debrisat fragment characterization system and processing status, 67th Inter- national Astronautical Congress, 2016.
Denise M Rousseau. Schema, promise and mutuality:
The building blocks of the psychological contract. Journal of occupational and organizational psychology. 74(4):511 1,
Meyer Scetbon and Zaid Harchaoui. A spectral analysis of dot-product kernels. In International Conference Artificial Intelligence and Statistics, pp. 3394-3402. PMLR, 2021.
Bobak Shahriari, Kevin Swersky, Ziyu Wang, Ryan P Adams, and Nando De Freitas. Taking the human out of the loop: A review of bayesian optimization. Proceedings of the IEEE, 104(1):
148-175, 2015.
Alistair Shilton, Sunil Gupta, Santu Rana, and Svetha Venkatesh. Regret bounds for transfer learning in bayesian optimisation. In Artificial Intelligence and Statistics, pp. 307-315, Florida, USA, 2017.
M Silcock, W Hall, B Fox, and N Warrior. Finite element modelling of metallic tubular crash structures with an explicit code. International Journal of Vehicle Safety, 1(4):292-303, 2006.
Niranjan Srinivas, Andreas Krause, Sham M. Kakade, and Matthias W. Seeger. Information- theoretic regret bounds for gaussian process optimization in the bandit setting. IEEE Transactions on Information Theory, 58(5):3250-3265, May 2012.
Mark Steyvers, Heliodoro Tejeda, Gavin Kerrigan, and Padhraic Smyth. Bayesian modeling of human-aa ai complementarity. Proceedings of the National Academy of Sciences, 119(11):
e2111547119, 2022.
J van Hoof, M Worswick, P Straznicky, and M Bolduc. Effects of post-failure modelling on the response of ballistically impacted composites. In Proceedings of the International Conference on Composite Materials, 1999.
Andrij Vasylenko, Jacinthe Gamon, Benjamin B Duff, Vladimir Gusev, Luke M Daniels, Marco Zanella, Felix Shin, Paul M Sharp. Alexandra Morscher, Ruiyong Chen, et al. Element selection for crystalline inorganic solid discovery guided by unsupervised machine learning of experimen- tally explored chemistry. Nature communications, 12(1):1-12, 2021.
Sarah C Watkinson, Lynne Boddy, K Burton, PR Darrah, D Eastwood, Mark D Fricker, and Monika Tlalka. New approaches to investigating the function of mycelial networks. Mycologist, 19(1):
11-17, 2005.
James Wilson, Frank Hutter, and Mare Deisenroth. Maximising acquisition functions for Bayesian optimisation. In Advances in Neural Information Processing Systems (NeurIPS 2018), pp. 9884 9895, 2018
Georgios N Yannakakis, Antonios Liapis, and Constantine Alexopoulos. Mixed-initiative co- creativity. 2014.
12

---

Under review as a conference paper at ICLR 2023
A APPENDIX
A.1 PROOFS OF REGRET BOUNDS
In this appendix consider a generalised version of the framework. For clarity (we favour brevity in the paper body, but clarity is essential here) we also require some additional notations, As in the main paper, our goal is to solve:
x
argmax f* (x)
xEX
where f* is only measurable via expensive and noisy process:
y = f (x) +V
where v is E-sub-Gaussian Gaussian noise.
Let's assume that have series of experimental batches 1, where batch contains P human generated experiments and ÿ Al generated experiments, giving a total of p = p experiments. We use a hat with an index i E M to indicate a property relating to human i, and a breve with an index j E M to indicate property relating to AI j, where:
M = Ng
(set of humans)
= Ng (set of Als)
We assume the batches are run sequentially, and wlog that the experiments within each batch are nominally ordered, so the set of all experiments may be indexed with We bar differentiate between a property indexed by experiment number which is unbarred (c.g. c²), and property indexed by batch number s, which barred (e.g. d,). For simplicity we define:
8; =
(batch in which experiment occurs)
P (s
1) +1
(first experiment in batch 8)
(St 1) +1
(first experiment in batch 8c)
L = (8
1) +1
(first human experiment in the batch s) (first human experiment the batch 8c)
= p (8t -1)+1
f ( 11)+1+p
(first AI experiment in the batch s)
(st 11+
(first AI experiment in the batch s;)
T. N, (s-
(set of experiments in batch s)
gg Nap +1
(set of experiments up and including batch s)
We assume that humans and AIs maintain batch the posterior means and variances are, respectively:
model that updated after each batch. So, after
Pr, (x) = ,x ki (x),
0 (x) Ki, (x.,xxxxx (x,x) x)
i,a (x)
Kis + +041
2 k (x)
P3. (x) Cj.kk, Kj.0 (x),
00 (x) -x,(xxxx)
kj. (x)
-T
j.o + 8 +0'1)
-1
kj,0 (x)
where:
òi K + +
K ++07 F.
8..
K = Ri (xi,
K = Kso (xe, xv) I.EET
k. . (x) Ri (x,, x)
ky. (x) Kso (X,, x)
1ETg.
LETS.
y. = fr (xx) + 1
F. = f (xx) leeTes
We also occasionally use:
y: 1 (xe) + ae les f. = f (xv) l
As this a batch algorithm we are concerned with the instantaneous regret for the batches, not the individual experiments therein. The instantaneous regret for batch is:
min Tt, = f (x+) (xx)
tET.
13

---

Under review as a conference paper at ICLR 2023
where is the instantaneous regret for experiment t. batch is:
The cumulative regret up to and including
Rg = Σ F eENg-1
We do not assume f drawn from any of the GP models for humans or Als, so the problem mis- specified. So, borrowing from Bogunovic & Krause (202I), assume humans and Als attempt to maximise the "closest" (best-in-class function to f in the respective hypothesis spaces, using the shorthand AA. Hk.." H,,s H. k,.""
H. = =f f Ao. Il I B,
M I Hijs |1/ S B3}
where the closest to optimal approximations of fr in the hypothesis spaces are:
Fhe argmin ||ʃ - 1" J argmin ||ʃ - -----11 1l
ʃεa...
felt,
As is usual in practice, kernels may be updated when the GP models are updated, typically using max-log-likellhod for Als or something more radical for the humans, which modifies the corre- sponding RKHSs. The difference between the best-in-class approximations and f* for batch assumed bounded as:
E/1
After each batch 8, we have the (nominal) GP models built on (nominal) observations of ft, and f. which have the same variance as the (real) models (the posterior variance is independent of y) but different posterior means:
a (x)
#T 0 ki,s (x),
## P3 (x)
Oz 2j Kjs (x)
We assume that test points are generated, either nominally (for humans) or directly (for Als), EC- GP-UCB style (Bogunovic & Krause 2021). from sequence of interleaved 3, sequences by the relevant human t iif e M Al - if E: MI in batch st, so:
0 if = E M 33g if = t i M
B.
Ei if = t iE M
Eee if =t - i M
Pi,n-1 (x) if = - E M
Pr- (x)
11 (x)
if t { MI
0 (x)
if = t E M
(x)
0 (x)
if = - 1 MI
using the acquisition function:
Oi (x) 14- (x) + (VB, o,- (x)
Generally we cannot control the human's exploitatiom/exxloration trade-off sequence B but we assume humans are conservative, so the sequence may be assumed small. We use the Al trade-off sequence 3,.. which we do control, to compensate for the conservative tendencies of the humans involved. We do however assume that the humans include at least some exploration in their decisions on the understanding that their knowledge is not, in fact, perfect, so 3.c 2 0 (note that a human who over-estimates their abilities may fail to meet this requirement, care is required to avoid this). For reasons which will become apparent, we assume that:
Bes, E (Gint Eist&ee) Bra. E (EreLEsos S
14

---

Under review as a conference paper at ICLR 2023
where Si..l, $ 1, which we will see makes the humans over-exploitative, and En, 2 1, which we will see makes the AIs over-explorative, and:
2
#
Li.
Σ
2ln (1)
R/,0"
1,8
Vo
2
#
V
2
(3) + +1+ 93
+
where V5.5 and 7j.. are, respectively, the max-information-gee gain terms for humans i M and Als j E M, as will be described shortly.
A.2 NOTES ON MAXIMUM INFORMATION GAIN
We are assuming p GPs, each of which will have different information gain that is a function of its kernel. All have the same dataset and get updated at the batch boundary. Thus, after S batches, if consider human i E M:
7, Sysp: (sp) H (y 1n
In 2TTee" Isp '
Hf (y 1) + H (Jspp ((s-1)p) 1 In 2Tee 2
1sp
= II 2)) + H (Usppy((ssn)) 1)p) hp)++ + H (USp- ((S 1/2 In 2nee F Isp
H (y(--1))))) + Σ H v In 2 Isp
tETs
H (y(s-2)p) + 1ETs
Σ H (vlly-1))
E H (BelYt-1) - In 2xeō Isp
sETs-1
Σ
H (v:ly:-1) 1) In 2nee Isp
1ETgy
2/2 Σ In 2Te -² 70 2-2
0 (x1)
In 2
lETgs
2 Σ In (1+ 1
2.2
i--1 (x1)
teTgs
where we have used X1..... Xap are deterministic conditioned on (s-1))p- and that the variances do not depend on y. The derivation for Al is essentially identical. In summary, therefore, the information gain is:
-000 i. (ysp fsp)= = Σ In 1 3 0,,--1 (x:) S fi
lETgs
I (ysp fsp) = Σ In
-22 ( د j4-1 (xi) S 3,35
1ETes
where Irs and R are, respectively, the maximum information gains for human and AI j over S batches.
A.3 NOTES ON THE HUMAN MODEL.
We posit that every human i M has an evolving model of the system:
f. (x) = Ii.o (Pi.e (x))
where Jr.. is some sense "simple" and P... E" R This fits into the above scheme
we let the form of 9.. dictate the kernel K,.. For example if we know that ÿ., linear- i.c. the human is known to be using some heuristic model of the form:
Pr. (x) = Wi. (x)
15

---

Under review as a conference paper at ICLR 2023
then use GP with a linear-derived kernel: Ri,s (x, Pi,s (x) Pi.. (x')
Similarly if the human is using a model 9i.s that can be captured by a -orderrer polynomial model:
f.. (x) (x)= w (V.) Pio (x)
gENai1
then we can use GP with a polynomial-derived kernel:
i., x') = (1 PT,. (x) Pi., (x))
Alternatively, if Rt. is more vague (i.e., the researcher knows that the factors P..,, are important but not the exact form of the relationship) then we might use a GP assuming a distance-based model:
(x,
(x, x') = exp (-2) Pr. (x) Pr.o (x211(2
similarly generic model that captures the worst-case behaviour of the human, along, hope- fully, with some insight into the thought processes used by them.
With regard to maximum information gain, because the human models evolving, m. will change with s, so it is convenient to define define fugtt maX m, to capture the worst- case feature-space dimensionality over S batches. We can then bound the maximum information gain for the human the worst-case of these models over all S batches. So for example, depending on the specifics a we have (Srinivas et al. 2012 Scetbon & Harchaoui 2021):
Linear: fis O (In (Sp))
Polynomial:
R. = O (In (Sp))
Squared-Exponenteee
Rr. O (In (1 (Sp)
In general we assume that the asymptotic behaviour of the human maximum information gain con- verges more quickly than that of the machine models. This makes intuitive sense of the human is applying linear polynomial heuristic model â.... which is captured by linear polynomial kernel, while the machines use more general GP models with SE or Matern type kernels (as is com- mon practice). Thus in this the human has better behaved maximum information gain the cost of a potentially non-zero gap ét, (presumably trending to 0 as the human gains improved in- sight into the problem and evolves their model better match the problem), while the machine has worse behaved maximum information gain but zero gap (assuming universal kernel like an SE kernel).
A.4 MATHRMATICAL PRELIMINARIES
We use p-norms extensively, where:
| (z.))))) |1ll,, (8x (x)l dx))
where R
where RRR" + R
with the extensions:
||x| 00 max, [
||x|| -00 min; [xs]
|| = inf (C> 0: (x) (x)
C for almost all x E Ra)
|/ sup (C20
C for almost all RT)
This is only a norm for p [1, oo] (though we may occasionally refer to as such loose sense), but is well defined for p E [00, 0) U c0) is not difficult to see that:
|x|| 2
| pr
Vp. p' E (0, col.
x
x pr Vp.p' E [-00, Sp
x
x p vp E o0), E -x)00
16

---

Under review as a conference paper at ICLR 2023
(the final inequality follows from the first two, noting that min; |zy| max; [ay), and similarly for functions); and furthermore:
| S n | AI psspsp S
when x E R.. We also use the following inequalities (o is the Hadamaard product):
xx x' O
I ||
|X|| plr.....
xx"
General. Hölder inequality
ffo
1" I|,
I llfll If p'l......
p
|x O x'Il
IV I
|
x
Reverse Hölder inequality
Ilf 3 f'll
Ifll
where
1 /
pm 1 and r,p, ;-.., tM
(0, oo), with the convention 0: and
E (0, 00).
We also use generalised (power) mean
PPčarić
1991
Pyun
1974). defined as:
min;
if 0 = -00
(+ a
if 0 E (-o0, 0)
Ma (1a), 02 -
(II 0.)
if A 0
( (9) max, Oi
if E (0, oo)
if = 0c
where 8 oo]. For example 1, 0, correspond, respectively, to the harmonic, geometric and arithmetric means. Note that, for all 0, 0' E 1-x0, 00), 0 < 0':
Ma (fa1, 02, .22...... ..,)....))
} S Mg (1a1, 02, an))
With a minor abuse of notation, we define:
Ma (a) Ma (fa1, a2,... an))
for a E RR", so that, for a E IR, we have the connection to the p-norms:
n |all a))
if E [-oo, 0) U (0, oo]
Mo (a)
i
The following result central to our proof
Lemma 2. Let b E RR, E [-00, oo],
q. E o0), /
0.
Then we have the
following bounds on M (a b):
M
M_ (a) M2 (b)
M
a O
b
M [g]
(a) M = (b)
M M- a
b)
IN M |2| (a) M |=| (b)
Proof. Let us suppose that, unlike the theorem, 2 E (-),,))L(O,,)))) 20). Then, using the generalised mean inequality:
M (a ( b) < Me (a l b) -==
bll n- ||a
+ b 1
If > 0 then, using Hölder's inequality:
M-x (a @ b) = nt
a 0
b 0
< n
n
= n -
12
1
z0
n [
a
|
[
|
= M xgl (a)
M
|200
(b)
"This result may be well known, but we have been unable find it in the literature.
17

---

Under review as a conference paper at ICLR 2023
If < 0 then, using the reverse Hölder inequality (noting the negative exponent):
M- (a @ b)
12 S
a
a
n FT
n T
|a|
M
is instructive to let l21/q. Then the most recent bound becomes:
M- b) - M-, (a) M ,2, (b)
which the limit q + oo simplifies to:
M- (a S M_, (a) Mr (b)
Finally, using the definitions:
M- (a S Mo (a b) My (a) My (b)
completing the proof.
A.5 BOGUNOVIC'S LEMMA
We have the following from Bogunovic & Krause (021))
# | (x)
Me- (x) S Vzo-- (x)
This follows from [Bogunovic & Krause 2021 Lemma 2) using that all models satisfy this result, and that the model is posterior the observations up and including the previous batch. It follows from this and the definitions that, for all x E X:
P (x) fr (x)
<
p i 1 (x) - #
fi (x)
+ Lite Vt,b..-1 (x) + Eie,
(10)
1 fr (x)l - 1o (x) - (x)
(8)- (x)
+ fit Vt,ō (x) El
A.6 THE REGRET BOUND
We hegin with the following uncertainty bound, which is largely based on the bound (Chowdhury &
Gopalan 2017
Theorem 2) and analogous (Srinivas al. 2012 Lemma 5.1) and (Bogunovic & Lemma 1):
Krause 2021
Lemma 3. Let (0, Assume noise variables are Sub-Gaussian. Let:
2
E
2ln
(1) +
Ri.0
V
1,8
(1I)
x#
V
2ln
(+) +
3,,0 +
5,0
18

---

Under review as a conference paper at ICLR 2023
Then, for all i E M and M.:
Pr ss, Vx X, li (x) fr (xx) ....
(V (18,0 + Cage vts) 0;,0- (x) + Ehg.
2 21-0
Pr Vs, Vs Vx E 1 (x) (x)) S ...
(V + Site
3 Vts
(x) + E (x)
21-6
Proof. We start with (Chowdhury & Gopalan 2017
Theorem 2). This states that, in our setting,
with probability 2 1 -d, simultaneously for all
and D):
A A (x)
fi (x)
S 0 i (x) v &,.0
## A (x)
f, (x)
f (x) V 1x10
4-1
for Re Xj.0 as specified by 11 Next, recall (0):
| (x) - f f (x) S
# P (x)
fi (x)
##
+ ワ
cيs VE,ōr.e-1 (x) + &
[ ( (x) S
# p (x)
5, (x)
*
+ Sife V.o (x) + Ej
and the result follows.
Remark: Alternatively, following (Fiedler et al. 2021 Theorem 1) we can use:
2
Lie 2
2n (+) + del
Ki + 031)
+
vo
81,0
2 2n (t) + In det
x K (K,.-1++01]
j.a
Using the uncertainty bound above, we obtain our first bound on instantaneous experiment-wise
regret based on (Srinivas 2012 Lemma 5.2) with some techniques borrowed from Bogunovic & Krause (202)):
Lemma 4. Let (0,1), Assume noise variables are Ssub-Gaussian. Assume that:
B,,, E (SiaL SioTXie)) Bis. (EioLisii (11o
where:
Σ
2 (+) + 9c.e
VB
2
E,.0
21n (1) + 11 5,,-
Val
Aeg
Then, simultaneously for all 2 1, the instantaneous regret is bounded as:
Ti < Pret
1 1--VGns
V (x*)
T $ fj +
1 v //,00 V (xx*)
19

---

Under review as a conference paper at ICLR 2023
with probability 2 1 8, where:
Pi,st =
+tyCrrt V Xiss + o vhe
fis
0i,--1 (t,+ts + 26
$1,0t = 2
1+
X2.0 + Eig
0 vt,
8,,0-1 (1 + +2
2
Proof. By the pretext and Lemma
fa (xx) S Pi (x*) (Viis + Euer
g vt.) م 1 (x*) + Cigs
with probability 2 1 6. By definition XG maximises the acquisition function, so:
Pi,a- (x') S fi, (xe,+)) + (vBne, +
=x Eule vE.) ō (x)++))
+ v Bus, + fige vts/ 0 (x+)
and hence:
fr (xx) fr (xxx+t) S Pi.ya- (xitts) + (V Bris, + Biis vtg dio-1 (x)+t) + tee.
Ris - V Bii, G (xx) - (xi,+))
follows that the instantaneous regret bounded by:
Ti S Po.o-11 (x2++)) + VVre. + Eiie 0 vEs :z,011 (Xt+te + Eie....
+ (VRes- / Buie, :ii-1 (xx) - 8zt (x (xi
So, using our assumptions on B we find that:
Ti S P (x()+)) + (VEnsrv 2ie + Euge vTs 8i.0-1 (xt+t) + tere...
+ (1-V V / (x*) - ( (x,,+))
Once again using our pretext and Lemma 3 we see that:
fr (xi++) 2 Pi
(g,+t) - (Vān. 12,0 + fide fige vts)
:i (xx,+)) Eie,
Hence:
Mi,ti 2 1ty/unt v Xe, Ripe vio SieeVio :i (x (xt,tz)
(1- (1-VZare)) (inee) Reodrro (x') + 2ir
and the desired result follows from the definitions. The proof for AI regret follows by an analogous argument.
To extend this usable batchwise instantaneous regret bound need to deal with the variance terms ox*), (x.==((x") in the above theorem. To do this, in the following theorem we use the generalised (power) mean, and in particular lemma 2 to split the batchwise risk bound into a constraint term containing the free variances and a risk bound that depends only on the various parameters of the problem:
20

---

Under review as a conference paper at ICLR 2023
Lemma 5. Fix ó and 0g [0, ool Assume noise variables are Gauss-aa and that:
Bug. (GaaddisrShothe
Br. S Sehtr
where Ei.os S.1, Ei 2 1, and:
2
(1) + +1+ 9i, +
i
Vo
X1,8
21n
(1)
+ 1 43.0
53.0
V
Algin
If:
Lesp
(Viio (1- v //una) 0o-1t) : E in uu..
exp
Ma.
I1
exp Visio (1- 1- (ioa)) )0-1) : E ni cnn
Then, simultaneously for all s 2 1, the batchwise instantaneous regret is bounded as:
F. S In (M_ (exp (For)))
with probability 2 1 where:
v Siat
V Ris + Btyle Vt,
ōi,--1 (xi,t:) + 2E
2
2
Est
jEM
1+ 1+ /uyu v 23,0 Legl Vto
2
83,01 (z,,t) 2ē
iEM
and:
0.-1t = max
max
( (x*)
max {õ (x*)
iEM
SEM
0,-11 min
min
( (x*))
min ( (x*))
ieht
jeMt
and exp is applied elementwise.
"The proof is for 0. [-00,00)) but the negative 0, case is of interest here.
21

---

Under review as a conference paper at ICLR 2023
Proof. It is convenient to re-frame the batch-wise instantaneous regret in log-space, and extract the max variance upper bound. Recalling that Si,44 S 1, S3,55 2 1:
2
V X1.0 + Sige o
Vie 0i,--1 11 (xt,+t) (i + t...
S min
min icM
Vinse) V
&eiorra a-- (x*) 2Erne
2
1- sint
V X3.x + fiie vLs
0- (i++t) +
min
jeM
V Sii
V
11-0 (x*) + -2E
3,01
exp
2
Xe.0
File b
VIa
0;,0-1 (xc++s)
<
In
min
min icMi
exp
V Ais
1
V 3 Sioel
-x-19
1
exp
Xj.5 +
Fide
Vt,
0 { + 22
min jEM
exp
Vi
1 //y00
Ōa-11
which may be re-written:
F. S In (M-x0 (
where:
b
a
à
RP b
b
E Rº
| 1esk
1+v See
02,0-1 (xz,);) + 2012s
cxp
2
2
V Xi,o
+
à
Vts
b
cxp
V
X
V (hal
Jz-17
+
NEM
cxp
2
1
1+ /+
2
Vxj.
Esuu Vt.
أ5,5-1 (xi,4s)
+22s1s
]
( V (Same)
6 =
exp
V Xxn0
0o-11
JEB
So, by lemma
we have that:
F. S In [M 8, (a) Ma. (b)) = In (M- (a)) + In (Ma. (b))
and by assumption Mj, (b) S 1, so:
F. S In (M (a))
and the result follows by the definition.
The next step is to convert this bound into a bound on the total regret. To obtain such bound we need some additional assumptions regarding "gap" parameters and the explorative/exploitttve nature of the humans and AIs. We do this with the following theorem:
Theorem 6. Fix 8 > 0 and 0, [0, co F
Assume noise variables are
and that:
B.e, C
'The proof is for 0. [-o0, 00], but the negative 0g case is of interest here.
22

---

Under review as a conference paper at ICLR 2023
where Ei.as S E3.2s 2 1, and:
2
Li.
Vā V 2 (1) +1+ + 91,0 +
1,8
Pr.
#
8j.5
v8 V
21n (3) + 1 + 9/ -
5/00
Pliia)
and:
exp
VVieo (1- 1--/une 0o-11)
: E M Mi U
Mr.
exp
V Vii
(1- v 5/i1)) 00-1))
: E Ema)
for all 8, where:
0,-1f = max
max
f (x*)] max { (x*)) jEM
iEM
8.-11 = min
min (ō1,--1 (x*)}
min { f (xx)
iEM
jEM
If f,,,,,, E3e = 0(ss ") for some E (g) 00) then, with probability 2 1 6:
1888 (Vies+o(u))
Rs $ in Mo
exp
i M U
2 Vits+0r1)
1
80
53, + +0 jE MI
where Čr = max,
8- R j,ot
and C₁22 = max,
8-2R3,m jutt
and:
In (1+ (iott)
In 1+ 2k k))00)
02 min 0,
sENs+1
x クイー
+EN++1 $j.8
max
Proof. Using the assumptions and Lemma for all 1:
5 we have that, with probability 2 1 - d, simultaneously
F. S.S In (M-o, (exp (fsr))) < In (M c. (exp (F.r))
where For O. Using the definition of a_
Σ
F.
Σ s1N
In (M (exp (fst))
sENs+1
In
II M_,, (exp (fst))
sENg+1
In
πI +1Ns+1
||exp (fat))l
ln
II p $
(exp
(Fxt))
1, 1
sENs+1
ln
11 Tt sENs+1
= ln
II 4 (exp (Fst))
sENg+1
23

---

Under review as a conference paper at ICLR 2023
Using the generalised Hölder inequality and the definition of M 0,;
Σ sENs+1
Fo S In
exp (Fat))
sENs-1
1 In
eexp (Fst))
+EN++1
In
+ NENg+1
exp
(Fot)
= In
M
4
exp (Fat)
+ENs+1
and so, again recalling that Fsr 2 0:
Σ +ENg+1
Fg
< In
# M
O
exp
(Tat)
s
sENg-1
In
M
exp
Est
*ENg+1
Next, using that X is increasing with s, and noting our restricted range in Ç, note that:
((Viss Big vla) ōi,--1
+ + 2
Fistt < 4
(i,+i)
<8max {
(VRiss + Eig vIs) 01
2
xi,+t)
1tye V XjS + 3o
#2 j.tt
<4
3 VTo
(x,++} +Eiits
{ ( v /Xyss + Regl
2
< 8 max
z vts
010-1 (xi
and so:
Eugs {(VEes +
Σ sENg+1
88 Σ max
vt.)
(x,,t)) Rig
1,st
sENst1
<
Σ (Vins+
tre VI.) 0 [X ti
+8 Σ E.
sENst1
sENs+1
2
1+V3 V Ejjs +
Σ *EN++1
< Σ max
Eidle
(x++t)): :G...
1.st
NEN++1
14y5 2
<8
Σ
Xj.5
Eiie Vls
x2 j, (x,+t)
+8 Σ 2
+1N
sENs+1
Recalling our assumption Exg,, E ((2))) < (Ç here is the Reimann zeta function):
O(s 9) for some q > y we find that, EvEN J/.t
Σ
<8 Σ (Vi.s+
202
02 +1(1)
rist
NENg+1
+1
sENs+1
3
Σ
j.st
<8 Σ
1
1ty
2
22
1,0-1
(Xi,tz) + +1
+1
+1N
24

---

Under review as a conference paper at ICLR 2023
Now, by the standard procedure:
Σ
0 (x),--) SCi² Σ In
1 + 8 -000 i (x),
+1N++1
+ENs+1
$ Ci Σ In 1 +
22
0 1,00111 (x+)
0
leTge
$ Či,29i,s
Σ
-22
0 j (x(,+s)
- و/22 Σ In 1 0
2
0 j,a-1 (xg
sENs+1
sEÑs+1
- U2 Σ In (1 0 0 j. -1 (x1)
-2
tE
where:
Či22 = max
2R
.ENs-1 In 1++ R 1.o
max
sENs+1 In [1+8 Ks
and so:
Σ
fist S 88"
(v (Viis Chafis+0(1
*ENg +1
2 1-v3m ( Vx,,+ +
Σ sENs+1
22
j,st I _
Recalling that
||- 1111/1 s VSI|
ll2 (in S dimensions):
Σ Pratt S V *1N++1
882
(VRis+0(1)) O:2fus+
Σ jot S vs, sENst1
1872
(Xis+001 {=
1
C3 (1)
and So:
Σ Piiot E iyot...
sENs+1 sENst1
< In
M á exp
VS "8² (0(1) + VRnss)
C + 011 i M U...
1+ (11) +
VS
2
/ /K5.s
C j M
2
Finally, noting that, for a > 0:
In M og (exp (exp(a))
In
M (exp (a))
In
p E, (ea:)
-8,
In
[ (eta)
In
M ū2 (exp (fa))
25

---

Under review as a conference paper at ICLR 2023
obtain:
E
Prat +
Σ Frot
+ENs+1
*ENs+1
< In M-
86² (0(1) (iss) + + 011) i M ju...
exp
V8
1455 (11)+
80²
Viss) 2
S j M
v
as required.
L
Finally, we consider the bounds the posterior variance at x*. We have the following result: Lemma 7. Fix j' E M. We have the bounds for E M. E Ml:
6 Lo) δi, (x*) S 6
j.s (x*) S j,st
j,sb
where:
8 +2psDD.... ... pa B²
++2
K.
(,8)
d' 3'tpsk 1,
1,8
8² 2'42paDi... Rer
13
u+2
j,8]
tpss 500
K
j,z
82 D
Exere, Di
++2
Rio
Ki,
1,81
8 t
8 42
Dymn Ri
Eierg, Bann
2
Ki
3,8t
8 tptpak 3.
and:
K Ris (x",x") x²),
j,,- K ja (x+,x" ,x))
Ristt Ris (x*,x1) x(): ;
K ji = K j.n (x" (x)
Kis,,tt Ris (xe, xx),
K jiss,,t j.o (xe, X1)
and:
Kistte Ki
244 Drsds
K Kistt
Kj
24
Di,,,1 min 1ETgo
Dis.. min 1E3go
(Dss))
Proof. Using the definition of GP posterior variance and K bounds we can minimise the posterior variance within the constraints given as:
0,0
(x*)
K 1,8
2
i.
IT ps
(
Rist IT + +
ps
k²
K.
L.tt k 82tpaki.. 2.07
Da
1,8
pa R s.u.t.
K
iss
8 K +psk soot tpsk (,
K i.
pokk i.estic
82
PRR Ris +pak Ri
83 R tpsk K -psk... t2pak Dive. psD
K i,
#t tosk Kookk Riz
#2 Rtr Di
K
1,8
8 g
26

---

Under review as a conference paper at ICLR 2023
and likewise for the Al posterior variances. For the upper bound, may pessimise the bound by first using the maximum eigenvalue:
LieTe, Kunt
82 se (x*) & Ri,0 Amax (Ri.) +0²
and by Gershgorin's 's circle theorem Amax (Ki))) S ExETEO R Rissttotus S0:
EnETgo Kiate
$2 3,5 (x*) S Ki, + Rroissts
82 "R EEnTe, Riaitneo Rij0- -Eire, R Riniee Ki,0
&R Rin
Now, we have that Kieet.x Ki Diio, where Di 10, Ko,)
8 82 R Rindent K,, peRii 2002 +2k Liete, Disn-EErres Dune 0,,0 (x*) S
Ki
8 + +ECcTe Riadadkii 8 8K + EiETe, R Rindent R,, pas +2R,, Eietg, Braw Energe
Br Kis
+ Rinwsaa Kis
8 *Ki + +icTe, Rid paki +2ki, Exete, B Eicre, Bran
Kis
Ok Evees +
Kis
82 Ki' +2k EnETe Dioe DiET Br 2** 8² 8² tpsk 112 8 +2 Bu Rir Energa Diod A Ki
Kig 1,0
8 +ppoki
and likewise for AI variances.
In this theorem the sequence Di,..- = -x),xxx))))<)x.xxx ixx,, x:)l 2 0 proxy for convergence, being minimised for x, = x* and increasing as becomes (a-posterior) less correlated with x If we consider the case considered in the paper namely human and AI with zero gap and a trade-off' sequence meeting the conditions of GP-UCB then know that the AI, operating alone, suffices to ensure convergence; and moreover adding additional observations (the human recommendations) will not prevent this. From here, it is not difficult to that the upper and lower bounds in the above ** theorem converge not just to ( but to one another, and if we further assume that K K j.a then the ratio of any upper bound on the posterior variance to the lower bound on any posterior variance will convergen to 1.
A.7 ADDITIONAL SYNTHETIC EXPERIMENTS
In addition to the experimental results mentioned in the main paper, we provide the optimisation performance of our proposed BO-Muse framework on the following synthetic benchmark functions: (a) Matyas-2D, and (b) Rastrigin-5D. The details of the additional synthetic optimisation benchmark functions are mentioned in Table 2
We maintain the same experimental settings mentioned in the main paper for the additional experi- conducted. For given d dimensional problem, we d+1 initial observations and optimise for 10 x d iterations i.c., the budget allocated for our synthetic experiments is set to 10 x d function evaluations. The simple regret plots obtained for Matyas-2D function and Rastrigin-5D function after 10 runs with random initialisations are shown in Figure 4a and Figure 4b respectively.
A.8 ABLATION STUDIES
A.8.1 VARYING DEGREFS OF EXPLOPTATION EXTEIITTIOOOEEPOOOATTTOOOOOO
We study the sensitivity of the exploitation eepptioattellooatiinnn parameter (3.) in our proposed BO-Muse framework and compare the optimisation performance. We vary the exploiaation-exploraiii param-
27

---

Under review as a conference paper at ICLR 2023
Functions
(*)
High Level Features
2 = 2 2/2
Matyas-2D
0.26(zf + x3) 0.48(21122)
Izz =1122
= Vi Ns.
Rastrigin-5D 10d + Σ [0 c0 ViE N₂
+/-3 cos cos Vjj E No
Table 2: Synthetic optimisation benchmark functions. Analytical forms are provided in the second column and the last column depicts the features used by a simulated human expert.
Matyas Functian
Restrigin5D Function
80
60-Muse 4 Humaan
B0-Muse + Human
6-
Genaric
Generiz
Simulated Human
Simulated Human
Simulated Human PE
Simulated Iluman
Simple Regret
Simple Regret
3-
50-
2-
1-
30
12
Function Evallationn
Function Evaluations
(a)
(b)
Figure 4: Simple regret vs iterations for synthetic Functions:(a) Matyas-2D (b) Rastrigin-5D
eter (f.) in the exponent range [0,3] i.e., A, E 10 to cover the whole spectrum from over- exploitative experts (B, = 1) to over-explorative experts (3, = 1000). We have tuned the Squared Exponential (SE) kernel hyper-parameted of the inherent GP surrogate models using maximum- likelihood estimation. The empirical results obtained for various synthetic functions are depicted in Figure 5 It is evident from the empirical results that BO-Muse teamed up with an expert following more of an exploitation strategy (3, = 1) has better convergence when compared to its counterpart teamed with pure explorative expert (9. = 1000).
A.8.2 BO-MUSE WITH EXPECTED IMPROVEMENT ACQUISITION FUNCTION
We have conducted an additional experiment to study the behaviour of our BO-Muse framework with different acquisition function strategies for the human experts. Expected Improvement (EI)
Ablation Study Acbley Function
Study Levy Fundtion
Ablition Study Rastrign Fanction
1
190
25
A=-1
.... 1,00
10
1000
2000
1300
Strpe Segret
Simple Segret
Smme Rengek
12-
Functin Evaluazions
Function Evaluadons
Function Evaluations
(a)
(b)
(c)
Figure 5: Ablation study with varying degrees of exploration parameter (3.) obtained
for (a) Ac (b) Levy-6D functions (c) Rastrigin-5D functions.
28

---

Under review as a conference paper at ICLR 2023
Ablation Study Ackley Function
26
BO-MUSE Human(GP-UCB)
BO-MUSE+ HumannEE)))
24
Generic BO
22
Simulated Human
Simple Regret
Simulated Human PE
20
18
16
14
10
5
10
15
20
25
30
35
40
45
Function Evaluations
Figure 6: Ablation study with El acquisition function obtained for Ackley function.
acquisition function (illson et al. 2018 guides the search for optima by taking into account the expected improvement over the current best solution. If fx(x*) ) the best value observed, then the next best query point is obtained by maximising the El acquisition function (x), given by:
El (x) {
( ((2) + 0x(x) ((2)
ix), 0
0
ix)
2 (x(x) (x+
(x(
where (2)) and () represents the Cumulative Distribution Function (CDF) and the Probability Density Funetion (PDF) of the standard normal distribution, respectively.
In this experiment, the GP-UCB acquisition function used by the human expert in BO-Muse frame- work is now replaced with the Expected Improvement acquisition function. We compare this new baseline (BO-Muse Human (EI)) with BO-Muse Human (GP-UCB) and all the other compet- ing baselines. The empirical results obtained for the experiment with El acquisition function depicted in Figure [6] As expected BO-Muse with the El acquisition function still outperforms the standard basclines considered, However, BO-Muse with GP-UCB acquisition function has superior performance when compared to its counterpart with the El acquisition function.
A.9 ADDITIONAL DETAILS OF CLASSIFICATION EXPERIMENTS
We have considered two real-world classification tasks using Support Vector Machines (SVMs) and Random Forests (RFs). This experiment involves hyperparameter tuning of SVMs and RFs operating on the Biodeg dataset classify biodegradable and non-biodegradable materials. We used publicly available Biodeg dataset from the UCI data repository (Dua & Graff 2017). Biodeg dataset consists of 1056 instances with 41 features. We randomly split the dataset into 80/20 train/test splits. Each time a hyperparameter set (design) is chosen, the model needs to retrained and evaluated on held out set. The goal to reach to the hyperparameter set that leads classification model with the minimum test classification error.
We have created two groups (arms) with members (2 students and 2 postdoctoral researchers) randomly allocated in each group. Each expert in the first group teams up with AI as per BO- Muse, while each expert in the second group (baseline) tunes the classifier completely on their own. For each of the classification tasks, the two groups use the same tuning budget (3 random initial designs + 30 further iterations. The aforementioned real-world task is suitable for our case: (1) It easier find multiple human experts for this task as Al graduate students and post-doctoral
29

---

Under review as a conference paper at ICLR 2023
researchers have good understanding of classification (SVM and RF) models and understand how its hyperparameters generally influence the model fitting, (2) This task is familiar to the ICLR and machine learning community.
The simple graphical interface used by each participant shown in Figure 7a and 7b The interface shows accumulated observations of classifier performance function of paper-parameters, with the best results shown in darker shades blue. Interfaces for both teams are the same, excepting that participants working with BO-Muse see the Al-generated observation in the previous (#) iteration (indicated by V) in addition to the existing set of observed samples. Experts click on the plot to provide their suggestion on the next syper-paramette set evaluate. The co-ordinates the clicked point is used as the expert suggestion.
Enter Expert Suggestion #5
Enter Expert Suggestion #4
59.3
80
63.0
66.7
64.5
Depth
0-
20
pdd
4-
-4
-2
2
10
20
30
40
50
Gamma
Number of samples per split
(a)
(b)
Figure 7: Simple graphical interface used by (a) Team and (b) Team for both the classification experiments using SVM and RF classifiers.
A. SPACECRAFT SHIELDING DESIGN EXPERIMENT
Our third experiment is to team with expert design spacecraft shields to protect from impact by orbital debris particles.
1.1
EXPERIMENTAL PROBLEM
Here we consider the design of two- three-wall shield for protection against cubic steel pro- jectile impacting face on, normal to the surface of the target plates, at an impact velocity of 7.0 km/s. There exists no sfate-oftthe-an solution for such an impact threat, however for protecting against spherical aluminium projectiles in this velocity domain the state-of-the-art solution would be "stuffed Whipple shield" after Christiansen et al. (1995), consisting of an outer aluminium plate, inner layers of aramid and ceramic fabrics, and rear wall (pressure hull) of aluminium. US. Japanese, and European modules on the ISS all utilise stuffed Whipple shield designs (Chris- tiansen al. 2009). The design space is schematically shown in Figure g Design variables include:
(1) plate material AA6061-T651 (2211((AA)) ("AL.", 4340 steel ("ST"), Kevlar/epoxy ("KE"), and ultra-high molecular weight polyethylene ("PE"); (2) plate thickness 0.1 to 1.0 em in 0.1 increments; (3) plate spacing. S 0.0 em to 10 cm in 1.0 em increments, an; (4) number of plates, or 3 (i.e., the 'outer bumper' plate may or may not be used). Only metal plates (i.e., "AL" "ST"" may be used for the 3rd plate. The full factorial design space includes 577,365 options.
A.10.2 BACKGROUND
Spacecraft are subject to impact by natural micrometeoroid and man-made orbital debris particles, collectively referred to space debris, during their orbital lifetime. The impact of such particles (typically at velocities above 10 km/s) is a significant risk to the safe operation of spacecraft and the fulfillment of mission objectives. Indeed, for manned spacecraft such the International Space
30

---

Under review as a conference paper at ICLR 2023
Plate (auter bumper)
Plate 2 (inner bumper)
Plate 3 (rear wall, metal only)
Projectile
V = km/s (fixed)
0 deg (fixed)
tbb
S
S₂
Figure 8: Schematic of the spec debris shield design problem. Our objective is a design solution that will prevent perforation of the spacecraft hull (Plate 3, rear wall) by modifying the plate materials, plate thicknesses, and plate spacing (S).
Station (ISS) and Space Shuttle Orbiter, space debris impact is the top mission risk (see e.g.. Hamlin al. (2013)) As such, all manned spacecraft and some robotic spacecraft carry dedicated protective shields. The most common shield configuration is simple design known Whipple shield with two thin plates separated by a gap. Meteoroid and debris particles, upon impact with the outer plate, fragment into cloud of solid, molten, and vaporised particles. This debris cloud expands as it propagates through the gap, resulting in a dispersed and substantially less lethal load upon the spacecraft hull. Multiple variants of the Whipple shield exist, including stuffed Whipple shields which utilise intermediate layers of high-strength and high-impedence fabric (see e.g. SS-CChistiianeenn al. )1995) and multi-shock shields which induce multiple impact shocks into the projectile to promote maximum melting and vaporisation (Cour-Palais & Crews] 1990).
Spacecraft debris shields are typically designed using a combination of semi-analytical equations, numerical simulations, and experimental testing. Simulations are performed in either explicit finite element solvers. e.g., ANSYS LS-DYNA, or shock physics solvers, e.g., CTH from Sandia National Laboratory. Modelling hypervelocity impact in those simulation codes requires substantial exper- tise to accurately projectile and target kinetmatics together with material response. Furthermore, such simulations can be computationally expensive, requiring hundreds of CPU hours depending on the geometric discretisation of the model. Experimentation is typically performed on laboratory accelerators known two-stage light gas guns. The number of such facilities that perform experiments with millimetre and centimetre sized proejetiles up to impact velocities of 7+ km/s is also expensive, costing the
very limited (estimated to be 20 globally). Such experiments order of thousands dollars, with a low through-pus -utt of approximately 1 experiment per day. In the design of space debris shielding, in order to minimise the number of experiments and simula- tions required, space debris is typically simplified to spherical aluminium particles. In reality, of course, the debris environment consists of range of materials, both metallic and non-metallic, the properties of which influence their impact lethality. Similarly, for robotic and manned spacecraft the majority of impact risk represented by millimeter-sized objects, the majority of which are fragmentation debris that have been generated by catestrophic breakup of a satellite or rocket body and are thus highly irregular in shape, see Rivero et al. (2016). Until recently the engineering envi- ronment models used to predict mission risk to space debris impact have also simplified the debris population as spherical aluminium objects, thus there was little incentive to introduce the added complexity of projectile shape and material effects in shielding design or characterisation studies. However, recent improvements in orbital debris environment engineering models, e.g., ORDEM 3.0 (Krisko 2014), and planned improvements to debris population source models, e.g., via DebriSat (Rivero et al.] 2016), aim to address some of these deficiencies. Shield design and characterisation, therefore, must also begin to account for projectile shape and material effects.
A.10.3 DETAILS OF EXPERIMENT
The spacecraft shielding design experiment utilises synthetic data generated via numerical simula- tion. This section provides additional information the simulation setup and evaluation. Simula-
31

---

Under review as a conference paper at ICLR 2023
Parameter
Value
Units
Source
EoS: Gruneisen
Density, p
2700
kg/m²
(Anon.
2003
Shear modulus, G
26.2
GPa
(Anon.
20033
Elastic modulus, E
68.3
GPa
Anon.
2003)
Poisson's ratio, V
0.33
(Anon.
2003
Melting temperature, T.
930
K
Corbeti
2006
Bulk soundspeed, co Parameter S
5240
m/s
Corbett
2006
1.4
Corbett
2006
Parameter S
0.0
Corbett
2006
Parameter Sy
0.0
Corbett
2006
Gruneisen gamma,
1.97
Corbett
2006
Specific heat, Cp
885
J/kgK
Corbett
2006
Strength: Johnson-Cool
Yield stress, A
324
MPa
Anderson al.
2006
Hardening constant, B
114
MPa
Anderson Jr et al.
2006
Hardening exponent, 71
0.42
Anderson Jr et al.
2006
Strain rate constant, C'
0.002
Anderson al. 2006
Thermal softening exponent,
1.34
Anderson Jr al.
2006
Reference strain rate, 60
1.0
/8
Anderson et al.
2006
Failure: Johnson-Cool
Failure constant, dj
0.0
Anddrrso Jr ct al.
2006
Triaxiality constant, d2
1.11
Anderson al.
2006
Triaxiality exponent, d3
-1.5
Anderson Jr al.
2006
Strain rate constant, da
0.0
Anderson et al.
2006
Thermal softening constant, ds
0.0
Anderson al.
2006)
Table Constitutive model parameters for AA6061-T651 used in the numerical simulations.
tions are performed in the explicit structural mechanics solver LS-DYNA from ANSYS (Hallquist, 2006). Simulations are performed in 3D using smooth particle hydrodynamics (SPH) discreti- sation scheme, which enables projectile fragmentation be modelled without arbitrary numerical erosion that would otherwise be required for mesh-based Lagrangian scheme. SPH elements of 0.05 mm diameter are used to disretise all simulated parts. The metallic materials, AA6061-T66 ("AL") and 4340 steel ("ST"", utilise Gruneisen equation of state (EoS) (Gruneisen, 1959). Johnson-Cook viscoplasticity model (Johnson & Cook 1983) and a Johnson-Cook fracture model (Johnson & Cook 1985), the constants for which are given in Tables and The aramid composite ("KE") is modelled as a continuum using the clastic-plastic orthotropic strength with failure model from LS-DYNA (MAT _059) and a linear EoS, the constants for which are given Table [5] The ultra-high molecular weight polyethylene ("PE"), specifically Dyneema HB26, is modelled using the orthotropic non-linear model and material constants from (Nguyen et al., 2016).
In Figure 9 series of frames from representative LS-DYNA simulation are provided, depicting the impact of the cubic steel projectile against a three-wall shield design.
Simulations are performed on AMD EPYC servers with 64 CPU cores (2.25 GHz) and TB of RAM. All simulations are performed parallel on 4 CPU cores and require 20-40 CPU hours to run, depending on the complexity (in this case thickness) of the target plates. The simulation models included a 3.0 cm thick aluminium alloy witness plate located 10.0 cm from the rear surface of the target rear wall (Plate 3 in Figure 8 to measure residual penetration in the event that the shield was perforated. Results were recorded as a binary pass/fail related to non-perforation or perforation of the target rear wall, respectively, together with continuous Depth of Penetration (DoP) measurement into the witness plate. Our objective is to design a protective shield that can defeat the projectile threat for minimal weight.
32

---

Under review as a conference paper at ICLR 2023
Parameter
Value
Units
Source
EoS: Gruneisen
Density,
7850
kg/m"
(Anon.
2022)
Shear modulus, G
80.0
GPa
(Anon.
2022)
Elastic modulus, E
205.0
GPa
(Anon.
2022)
Poisson's 's ratio,
0.29
Anon.
2022)
Mclting temperature, T'
1700
K m/s
(Anon.
2022)
Bulk soundspeed, co
3935
Banerjee
2007
Parameter S
1.578
Banerjee
2007)
Parameter S2
0.0
(Banerjee
2007)
Parameter S3
0.0
Banerjee
2007)
Gruncisen gamma, I'
1.69
[anerjee
2007)
Specific heat, Cp
475
J/kgK
(Anon.
2022)
Strength: Johnson-Coud
Yield stress, A
910
MPa
Hollqquis al.
2001
Hardening constant, B
586
MPa
(Holqqustt al.
2001
Hardening exponent,
0.26
(Holmquist et al
2001
Strain rate constant, C'
0.014
(Holmquist et al.
2001
Thermal softening exponent, m
1.03
(Holmquist al.
200T
Reference strain rate, Eg
1.0
is
Holmquist et al.
2001
Failure: Johnson-Cook
Failure constant, d1
-0.80
(Holmquist et al.
2001
Triaxiality constant, d2 Triaxiality exponent, d3
2.10
(Holqqust al.
2001)
-0.50
Holmquist al.
200T
Strain rate constant, da
0.002
Holmquist et al
2001
Thermal softening constant, ds
0.61
Holmquist et al.
2001
Table Constitutive model parameters for 4340 steel used in the numerical simulations.
Parameter
Value
Units
Source
EoS: Linear
Density,
1230
kg/m m
van Hoof al.
1999)
Shear modulus, Gn1
0.77
GPa
van
Hoof et a1.
1999)
Shear modulus, G22
1.36
GPa
van Hoof et al.
1999)
Shear modulus, G33
1.36
GPa
Hoof
1999)
Elastic modulus, En1
18.5
GPa
van Hoof et al.
1999)
Elastic modulus, E2
18.5
GPa
van Hoof et al.
1999)
Elastic modulus, Eg3
6.0
GPa
Hoot al. 1999)
Poisson's ratio,
0.25
van Hoof et al.
1999)
Poisson's ratio, Ma1
0.33
van
Hoof al
1999)
Poisson's ratio, 1/2
0.33
van Hoof
1999)
Strength: Composite Failure
In-plane shear strength, S21
9.0
MPa
Hoof al. 1999)
Transverse shear strength, S31
271.5
MPa
van
Hoof et al
1999)
Transverse shear strength, Saz
271.5
MPa
van Hoof et al.
1999)
Longitudinal compressive strength, Cn
221.0
MPa
(Silcock et al. 20066
Transverse compressive strength, C22
221.0
MPa
(Slloock et al
2006)
Normal compressive strength, Cys
1200.0
MPa
van Hoof et al. 1999)
Longitudinal tensile strength, Th
740.0
MPa
van Hoof
1999)
Transverse tensile strength, T22
740.0
MPa
van Hoot et al.
1999)
Normal tensile strength, T33
34.5
MPa
van Hoof et al. 1999)
Table Constitutive model parameters for woven aramid composite used in the numerical simula-
tions.
33

---

Under review as a conference paper at ICLR 2023
0.0 ms
0.008 ms
0.02 ms
0.07 ms
Figure 9: Series of frames from an LS-DYNA simulation showing the impact of a steel cubic particle (yellow) km/s against shield consisting of three 1.0 thick plates of AA6061-7651 (red outer bumper, blue inner bumper, green rear wall) with 3.0 mm thick AA6061-T651 witness plate (brown) positioned 10 behind the rearmost surface of the shield. The projectile is shown to fragment into dispersed cloud of projectile and shield fragments which radially disperse before impacting upon the rear wall. The rear wall is perforated and fragments are observed to crater the witness plate, providing non-zero depth of penetration measurement.
Three initial designs are evaluated by the human expert, the details of which are provided in Table together with the simulation results. Based on these results we assume that target designs with areal weights significantly less than kg/ /m2 are likely infeasible, while designs significantly heavier (> 15kg/m²) are not of interest. Within this weight range the design space includes 577,365 potential options. We perform the optimisation in iterative batches of size 2, with one suggestion from the BO and one from the human expert.
10. DIFFERENCE TO A CLASSICAL CS SETTING
We work with expert and this differs from usual CS settings where multiple experts perform the same task because:
1. Level of expertise required is high, and access difficult: The design of shields for protect- ing against space debris impact hypervelocity highly specialised discipline typically limited to national space agencies or their primary contractors. For instance, shielding onboard the ISS was predominantly developed by NASA and Boeing for US modules, ROSCOSMOS and RKK Energia for Russian modules, and JAXA for the Japanese mod- ule. Shielding on the Furopean Coloumbus module was designed primarily by Alenia Aerospazio under contract to the European Space Agency, but borrowed heavily from the NASA designs (see e.g. Destefanis et al. (1999). Therefore, recruiting multiple experts formidable task.
2. The cost of experiments high: Due to the cost and access limitations on experimental facilities, utilise numerical simulations for this design study. Such simulations dif- ficult to design and validate, our expert (with 20 years experience on such codes) required about 120 hours to build the simulation models. In addition, the simulations be com-
34

---

Under review as a conference paper at ICLR 2023
putationally expensive, requiring on the order of 100-200 CPU hours per simulation for moderate CPU.
3. The design problem hard: There exists no state-of-the-a solution for the defined prob- lem. The nearest analogue is shield designed for a spherical aluminium projectile, for which the state-of-the-art is a stuffed Whipple shield. This shielding configuration has been used to define our optimisation variables. Existing semi-analytical penetration laws, such as those in Christiansen et al. (1995), are not valid for application with non-spherical non-aluminium projectiles.
4. The expert cannot repeat the same design task as they learn during the first experiment.
All these factors mean that for a real experiment we can only show how BO-Muse helps a single expert for new problem for which there is no state-of-art solution.
A.10.5 RESULTS
Results of the experiment are provided in Table 6 We can observe that the human expert initially explored designs similar to the stuffed Whipple concept (i.e., metallic outer bumper, KE or PE inner bumper, and metallic rear wall, with the inner bumper being roughly located at the mid-point between the two metallic plates). By design ID the expert had identified a stuffed Whipple design that was successful at defeating the projectile, with an areal weight of 14.3 /n/m.. Between design IDs and 19 we can observe the expert exploiting this successful design to identify a lower weight solution, without success. Up to this point the expert does not seem to have been influenced by the BO suggestions. By design ID 21, however, we can observe the expert beginning to exploit BO suggestions, with design ID 21 modification of ID 5 and 14, design ID 23 modification of 18, and so on. Design ID 27, an expert suggestion that is an exploitation of the BO suggested ID 22, was successful at defeating the projectile, albeit at a higher weight than the previously identified solutions at ID 9 and 11 (16.9 kg/ sg/m²). The final expert design, ID 29, is a further exploitations of ID 27 intended reduce weight and is found be the best solution identified during the experiment, with an areal weight of 13.8 kg/m².
The BO-Muse design ID 22 and subsequent human exploitations (design IDs 27 and 29) are, ac- cording to our human expert, highly unusual configurations for spacecraft debris shielding that they would not have otherwise considered if not for the BO suggestion. The conventional design method- ology based on typical shields used in flight harware suggest that the outer bumper have density and a shock impedance comparable to that of the projectile and be sized such that the shock rarefaction and tensile release wave superimpose towards the back of the projectile, maximising fragmentation and radial dispersion. An internal fabric layer, such as that used in stuffed Whipple shields (see e.g., Christiansen al. (1995)) is then intended to catch and decelerate projectile fragments prior to im- pact upon the shield rear wall. This general design principle has been established through ongoing investigation since the Apollo program and matured for the International Space Station with signif- icant, proven success. Design ID 22 and the subsequent designs (IDs 27 and 29) are a substantial deviation from these established principles and at current it is unclear why they have been successful further investigation is needed.
Evaluating this experiment - the role of BO-Muse was to inject novelty in the design process, from which the human expert could take inspiration and perform exploitation. We consider this to have been successfully demonstrated in a real applied engineering design experiment
A.11 DISCUSSION OF LIMITATIONS
In this section we present a brief discussion of the limitations of our work. With regard to the human expert, we have assumed that "Cognitive entrenchment" behaviour occurs, which is backed by recent studies (Dane 2010, Daw et al., 2006). This may not hold strictly in all cases, which may cause the algorithm's 's sample efficiency to be lower than expected as BO may over-compensate for expected expert over-exploitation that does not eventuate. Similarly, we assume that the expert is able to improve their model the number of observations available to them increases. However a less skilled expert may fail to do this, and subsequently the algorithm's 's sample efficiency may suffer as, after a point, expert suggestions may cease to be useful. Finally, the human expert may
35

---

Under review as a conference paper at ICLR 2023
Plate ]
Gupl
Plate 2
Gap2
Plate 3
SC
Weightt (kgg/m
Resllt
DoP
Source
(cmn)
stem)
(caa)
stem)
(cm)
[1/b]]
(cm)
ST
0.2
2.0
ST
0.2
5.0
ST
0.5
7.9
7.1
2.89
expert
AL
1.0
AL
1.0
9.5
8.1
0.31
expert
10.0
13.3
0.01
BO
ST
0.99
0.0
AL
0.9
7.0
AL
0.66
9.4
11.1
1
0.43
exper
1.0
5.4
espent
ST
0.3
5.0
KB
1.0
2.0
ST
1.0
9.3
11.4
0.07
BO
7.0
0.9
A1.
0.9
9.3
13.4
expert
ST
1.0
14.3
10
BO
AL
0.99
0.0
KB
0.2
8.0
ST
0.5
9.6
6.6
1.90
export
10.0
13
10.6
4.0
KB
1.0
3.0
51
0.99
9.5
13.0
0,01
0.9
0.9
8.2
expert
13.0
16
DO
0.0
ST
0.9
9.0
AL
0.1
10.0
7.3
0.32
expen
0.7
1.0
ST
0.8
9.5
13.0
0.01
0.1
5.6
19
00.5
6.0
PE
1.0
1.0
51
1.0
9.5
12.8
0.06
0.9
expert
1.0
14.6
22
DO
KB
0.9
0.0
ST
0.9
8.0
ST
0.1
9.9
9.0
0.26
expert
1.0
A1.
1.0
10.0
11.8
0.11
0.9
25
ST
00.4
6.0
AL
1.0
1.0
S1
1.0
9.4
13.7
0.03
BO
0.0
0.1
ST
0.9
10.0
7.2
1.53
expert
10.0
16.9
28
B0
AL.
0.9
0.0
ST
0.1
8.0
A1.
0.9
9.9
5.6
1.75
expert
13.8
Table 6: Results of the spacecraft protective design optimisation experiment. The best results are indicated in bold (SC space claim).
36

---

Under review as a conference paper at ICLR 2023
not behave precisely like GP-UCB model would suggest, again resulting in lower efficiency. We note, however, that even when the human is unable to perform as expected, BO-Muse will still have sublinear convergence. In such worst-case scenario every second experiment will, in effect, be wasted. However the data from these "wasted" experiments will still provide additional observations of f for the machine's GP model, which only improve the model's 's accuracy. The behaviour of the algorithm in this case can therefore be analysed in the "machine running GP-UCB plus improved prior due additional data" regime that is, GP-UCB algorithm with Al-generated suggestions, an exploration parameter / that is increased by constant multiplicative factor, and a stream of additional (harmless or even potentially informative) human human-guided experimental observations which suffices to ensure sublinear convergence (Srinivas et al. 2012). We also note that the aforesaid worst-case scenario highly unlikely on the assumption that the human expert knowledgeable in the relevant field.
37


Reviews:
[{"review_id": "erH2qh1Pia", "reviewer": "Reviewer_XANP", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposed a bayesian optimization framework for human-AI teamwork in collaborative tasks such as experiment design and optimizing blackbox functions. The authors assume that humans are likely to over-exploit, and thus propose the AI's role as a source of exploration, and they validate their method on synthetic and real-world optimization experiments. ", "summary_of_the_review": "I am really excited by the problem this paper addresses. Unfortunately, I don't believe it's ready for acceptance in its current state (largely due to clarity and presentation issues), but would be very excited to see this work in a more polished form. While the overarching idea -- providing a formalism for incorporating human domain expertise in optimal experiment design -- is very interesting and timely given the exciting progress we have seen in recent years of neural networks aiding humans in creative tasks / search problems,  I believe the paper focuses too much on the synthetic settings and a rather dense analysis of the Bo-Muse algorithm, leaving room for only minimal real-world experimental details (e.g. the space shield design set-up is almost entirely in the Appendix, whereas I feel like such experiments should be thoroughly analyzed and discussed, even if that means being presented on their own in a separate venue). ", "strength_and_weaknesses": "Strengths: I overall really like the idea of bridging bayesian optimization / experiment design with human-AI collaboration, and think this is a promising direction. \n\n\n\nW1: The notation throughout the analysis could be significantly improved in terms of clarity and description. Even as minor as stating \"dataset\" before introducing \\mathcal{D}, \"kernel function\" before K (before Eq 1) would be good to have, as I think the strength of this paper could be its appeal to those who work in human-AI co-creative applications, and thus a more clear  and concise description of Bayesian optimization and GPs for such readers would significantly improve the paper quality (and since the paper's novelty is not on the BO side, and more an application-oriented, I'd argue such clarity is necessary). Sometimes variables are mentioned throughout the paper, and could benefit from reminders on what they are (e.g. max info gain \\gamma). It would also help to use more distinctive markings for AI / human suggestions (maybe super scripts AI and H?). \nFinally, the regret bounds in 3.3. feel quite isolated from the rest of the paper, and it's not clear what the central takeaway of that section is apart from just providing regret bounds -- without high level implications for parameter selection, it's not clear this is a more valuable part of the paper than, for example, experiment details that were pushed to the Appendix. \n\nW2: Assumptions of human model - The novelty of this paper rests in the application of BO to human-AI co-creativity / experiment design, so I wish there was more thoughtful handling of the assumptions being made about the human in BO-Muse. For example, does the assumption that humans are conservation reasonable for all tasks and settings? Why can we assume the number of features (m_s) in the human model is non-decreasing (a human may discover a major highly predictive feature only later on)? Why can the convergence of the human's max info gain be stronger than the AI's? Why are linear/polynomial models sufficient for the human? It would have been really interesting to even see real-world experiments that try to validate some of these assumptions.\n\nW3: Thoroughness of experiments - It would be helpful if the experiments in the main part of the paper had more thorough details provided. Were ablations over \\Beta done for Ackley and Levy functions, so we could get a sense of sensitivity to parameter vs. method? In the Appendix for the Matyas function, why does Simulated Human to better than the proposed BO-Muse method? How does sensitivity to \\beta change between tasks? \n\nWith the real-world human experiments, how do we know that the benefit of BO-Muse is not just due to humans receiving additional information, regardless of how well-optimized that information was for the task (e.g. a random acquisition heuristic)? \n\nMinor:\n- the template and font appears different from other ICLR papers \n- could the authors provide more discussion on the limitations of assuming f^star is drawn from the AI's GP?\n- there were quite a few grammatical issues , the authors should do another pass to edit for clarity ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "XL1b8aAVLUf", "reviewer": "Reviewer_xwm5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This work solves a problem of sample-efficient experimental design with teaming of a human expert and AI.  Since an AI starts to solve the experimental design from scratch, a human expert can help the AI to accelerate the design process.  To fully utilize such knowledge from human experts and AI system, the authors propose a batch Bayesian optimization technique.  Finally, they provide the experimental results that show the effectiveness of the proposed method.", "summary_of_the_review": "Please see the text boxes above.", "strength_and_weaknesses": "### Strengths\n\n* It solves an interesting topic that considers a realistic scenario with human experts.\n\n* The introduction section is very well-written and plausible. In particular, the motivation is strong.\n\n### Weaknesses\n\n* The proposed method is not novel.  I think that it is a simple combination of existing methods.\n\n* Regret analysis is limited, since the regret bound heavily relies on the bound by the AI system.  Since it utilizes the knowledge of human experts that controls how humans know the exact location of solutions, the bound is meaningless.\n\n* Baseline methods are not enough.  I think it should be compared to diverse existing baselines.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "2: The contributions are only marginally significant or novel.", "empirical_novelty_and_significance": "2: The contributions are only marginally significant or novel."}, "extra_text": {}}, {"review_id": "FwfYbEQjvK", "reviewer": "Reviewer_LH2r", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "The paper proposes a framework that combines expert knowledge and AI for improved experimental design. The principle is to let humans drive the discovery/optimization process while the AI provides additional data to overcome the strong exploitation bias of humans. The AI model consists of a GP that is fit using data from human + AI-recommended designs and employs a GP-UCB acquisition function. For simulation and theoretical developments, the human model consists of a misspecified model and employs an acquisition (EC-GP-UCB) function with parameters chosen to simulate human biases. The paper reports results on both synthetic and real-world experiments.\n", "summary_of_the_review": "Overall, the paper reads well and proposes a relatively novel framework. Although the technical novelty is limited, the paper analyzes the proposal, providing theoretical regret bounds. I have some issues, mainly regarding the fairness of experiments. Thus, I am inclined toward accepting the paper.\n\n-------\nGiven the limited time, I acknowledge that I have not checked the proof in the Appendix.", "strength_and_weaknesses": "Strengths\n- The paper proposes a principled way to incorporate expert knowledge while providing theoretical guarantees in the form of improved sublinear regret bounds wrt AI-alone BO.\n- The paper reports experiments with humans on relevant experimental design tasks.\n- I like the simplicity of the idea, which basically equips humans with AI (GP) that leverages their designs and helps them to explore outside their strong beliefs.\n\nWeaknesses\n- Some modeling choices seem arbitrary and mainly motivated by following existing works.\n- It needs to be clarified if the experiments are fair, e.g., if the baselines use the same number of data samples.\n\n\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Reject