Human-AI Collaborative Bayesian Optimisation
Authors: Arun Kumar Anjanapura Venkatesh
Santu Rana
Alistair Shilton
Svetha Venkatesh
Venue: NeurIPS 2022 Accept (2022)
Keywords: Human-AI Teaming
Bayesian Optimisation
Bayesian Learning
Classification
Hyperparameter Optimisation
Kernel Methods

Abstract: Abstract Human-AI collaboration looks at harnessing the complementary strengths of both humans and AI. We propose a new method for human-AI collaboration in Bayesian optimisation where the optimum is mainly pursued by the Bayesian optimisation algorithm following complex computation, whilst getting occasional help from the accompanying expert having a deeper knowledge of the underlying physical phenomenon. We expect experts to have some understanding of the correlation structures of the experimental system, but not the location of the optimum. The expert provides feedback by either changing the current recommendation or providing her belief on the good and bad regions of the search space based on the current observations. Our proposed method takes such feedback to build a model that aligns with the expert’s model and then uses it for optimisation. We provide theoretical underpinning on why such an approach may be more efficient than the one without expert’s feedback. The empirical results show the robustness and superiority of our method with promising efficiency gains.

Human-AI Collaborative Bayesian Optimisation
Arun Kumar A V, Santu Rana, Alistair Shilton, Svetha Venkatesh Applied Artificial Intelligence Institute (A²1²), Deakin University Waum Ponds, Geelong, Australia ( (aan janapuraven,,, santu. rana, a issaar raa.... shi lton, svetha. venkatesh (areesssh@deeeeee @deakin. edu.au
Abstract
Human-AI collaboration looks at harnessing the complementary strengths of both humans and AI. We propose new method for human-AI collaboration in Bayesian optimisation where the optimum mainly pursued by the Bayesian op- timisation algorithm following complex computation, whilst getting occasional help from the accompanying expert having a deeper knowledge of the underlying physical phenomenon. We expect experts to have some understanding of the cor- relation structures of the experimental system, but not the location of the optimum. The expert provides feedback by either changing the current recommendation providing her belief on the good and bad regions of the search space based on the current observations. Our proposed method takes such feedback to build a model that aligns with the expert's 's model and then uses it for optimisation. We provide theoretical underpinning than the one without expert's feedhack. The empirical results show the robustness and superiority of our method with promising efficiency gains.
why such approach may be more efficient
Introduction
The deep penetration of Al systems into wider society demonstrates its ability to deal with complex real-world tasks. However, there still exists plethora of complicated real-world problems (Roccetti cct al., 2020 Hechler et al. 2020] that AI systems are unable to properly address. In such complex real-world tasks, human-Al collaborative systems provide viable alternative. Through appro- priate collaboration strategy we can supplement the speed, scalability and quantitative abilities of Al systems with the reasoning and abstraction ability of human users and make the combination exceedingly capable [Carroll et al. 2019 Maadi et al. 2021
Recent studies, mostly proposed in supervised learning setting, used either Al requesting help of human experts if its prediction lacks confidence [Madras et al. [017]] or as guided by meta-policy that focuses the joint performance [Wilder al.) 20200] Mozannar and Sontag [2020) De et al. [2020] considered also the cost associated with human decision making while learning a joint policy. Bayesian optimisation (BO) Brochu et al. 2010), an efficient tool for global optimisation that can potentially play a much bigger role in scientific experimentation and product design, provides a rich playground for human-Al Al collaboration paradigm because the nature of applications ensures that there would be an expert running the optimisation process. There are some works on incorporating expert's knowledge in Bayesian optimisation procedure. However, nearly all of the work relies expert articulating a static type of knowledge in the beginning of the optimisation process either in terms of trends [Li et al. 2018 Rihhmāki and Vehtari 2010). shapes of functions [Andersen al., 2017 Venkatesh et al.] 2019 or suggesting a similar experimental system for transfer learning [oy al. 2016 Shilton et al. (2017. In reality, most of the time experts may not be able to reveal knowledee in terms of such explicit specifications. Expert's knowledge will also more likely evolve over the course of optimisation process through assimilation of the new data points. Unfortunately,
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

HAT
Ground truth
Probability Density
1.5
3.
1.0
2
1.0
x2
2
0.3
0.5
0.7
0.9
1.1
11
Lengthscale for 2'2
(a)
(b)
Figure 1: (a) Damped oscillator function (b) Hyperparameter distribution obtained using our pro- posed HAT based BO (blue) and standard BO (green). Red line shows the ground truth lengthscale.
a true collaborative system that takes into account such a fluid knowledge from the human expert via an intuitive communication interface so far lacking.
To address this problem, we propose new Human-Al Teaming (HAT) based Bayesian optimisation framework that takes expert input during the course of optimisation through one of the two ways: (i) correction of the current recommendation, or (ii) specification of a good region and bad region based on the current observation set. We believe these casier specify, than say, global trend or global shape of a function because an expert may be able to provide reasoning surrounding a small neighbourhood of sample point. Once such feedback is given, incorporate into Bayesian optimisation by changing the model selection process that ensures that such relative knowledge that the corrected recommendation is or the good region is more promising than the bad region are maintained. We use the acquisition function as the means to compute the promise and use the aforesaid feedback during model selection via maximum likelihood. The model derived from these constraints subsequently used for the recommendation. Our framework allows expert to intervene only when they are willing intervene and also allows them to evolve their knowledge over the course of optimisation.
promising
compared the current recommendation
Theoretically, show that when Gaussian process is used to model the function, this extra channel of information narrows the space of functions. We characterise such narrower distributions in terms of Sobolev spaces Tartar 2007] and show how it admits tighter upper bound of the total regret when an appropriate trade-off factor is used in the acquisition function. Further, we show in Figure la and [b] case of HAT framework standard likelihood based model selection of lengthscales of a Squared Exponential (SE) kernel. For a particular dimension (22) we observe that the HAT framework with expert having ground truth knowledge infers models that are distributed more narrowly than the standard one, inducing a smaller function space. We evaluate the performance of our proposed methods both synthetic benchmark functions and real-world datasets. We tune the hyperparameters of the Support Vector Machine (SVM) classifier [Burges] [998] Christmann and Steinwart 2008] operating on publicly available multi-ddmmensionl real-world datasets. We com- pare the optimisation performance of our proposed Human-AI Teaming (HAT) framework against different variations of the Bayesian optimisation algorithm. The empirical results prove the effi cacy of our proposed framework. We also provide small-scal. scale real-user study to demonstrate the superiority our proposed Human-Al Teaming approach over Al only approach.
Background
Notations. for Reals. X is an index set and E X. N,, (1,2.....,n)) U for the set union. is the determinant. We use lower case bold fonts for vectors. for 7th element in vector v. vT for the transpose of vector v. We use upper case bold fonts M (and bold Greek symbols) for matrices and M. for the element in row and 5th column of M. abs(.) for the absolute value.

---

2.1 Gaussian Process
Gaussian Process (GP) WWlllams and Rasmussen 2006] a non-parametrie model, that provides flexible framework for placing prior on functions. GP defines a distribution over the set of possi- ble functions given observations. Though there are other popular surrogate models such as Wiener process [Kushner] 1964] and Student-t process (Shah et al. [014]. GP is still the preferred model be- cause of its simplicity. A GP is completely specified by a mean function ju(x) and a kernel function k(x,x') The unknown objective function f(x) is modelled using GP as f(x) ~ (P(((())))))))))))))))(())) ((x,)))(xxx))) ((), denotes the set of t observations, then according to the properties of the observation (x,+1+ 1x and the set of previously observed data samples Dia are jointly Gaussian. Therefore, the posterior distribution for the new observation 1/++1 is com- puted as P(y++ [Di:t, Xx+11 ) = [((((x+1)) 1), (x
If Di:: = {x1=,> )
Gaussian process,
where p(x++1) [ + J I] J (xx+1) = 1(x(+1, X(+1) k + La T k = [k(x1, X(+1), kxx,, Xx+1)) Kij = k(X,,,, Xj) Vi, Vjj E Nr and o²x is the variance of the white Gaussian noise.
The kernel (covariance) function k x x X + R used in Gaussian process plays crucial role in modelling the Gaussian process surrogates. The kernel function incorporates our prior belief about the unknown objective function f. The kernel hyperparameters 0 are usually estimated by
maximising the marginal likelihood given as L = =plyy X,0) = Jpl f) p(f | X,0) df. The closed-form formulation of Gaussian process log-likelihood is given as:
log L - 2 ( (OKK+ 1)
1
1 2
log K + + I - t
log(2T)
(1)
2.2 Bayesian Optimisation
Bayesian optimisation (BO) (Shahriari al., 2013 Frazier 2018 has proliferated into many do-
mains, including sensor networks |Garnett et al. 2010 Zhang et al. 2018). robotics [Martinez- Cantin et al. 20077 Bossens and Tarapore| 2021] intelligent environmental monitoring [aarchant
and Ramos 2012]. and information retrieval Wang el al. 2014]. BO offers an efficient framework
for the global optimisation of an expensive and noisy black box function fxx, represented as:
x argmax
where X corresponds to a compact search space. The observations of the objective function f(x) are assumed be corrupted with a white Gaussian noise i.e., 3/1 = fxx) + et, where e, (0, d ). Bayesian optimisation models the objective function by placing a prior distribution over the space of unknown objective functions and then use it determine the next best locations to sample. Specifically it consists of two main components: (i) Gaussian process model (Williams and Ras- mussen, 2006), and (ii) an acquisition function [Kushner] 1964 Wilson al. 2018 The GP pre- dictive distribution obtained is used to select the next sampling location with the high promise of finding the optima. The acquisition function guides the search for optima by suitably balancing the explovation-exploiaaiio trade-off. [Srinivas et al. (2012) have discussed the Gaussian Process-Upper Confidence Bound (GP-UCB) acquisition function using the upper confidence bound selection cri- terion. A GP-UCB acquisition function at tth iteration is given by:
OGG. (x) = uu() + v Br (x)
where 3r is a trade-off parameter that balances between the exploitation and exploration. Chowdhury and Gopalan| (0177) have discussed the possible choices for 83 and their implications on the overall regret. In general, to achieve a faster convergence rate B, is increased with Ollog t). An iterative algorithm for the standard Bayesian optimisation is provided in the supplementary material (A.1).
2.3 Reproducing Kernel Hilbert Spaces (RKHS)
The kernel function used in GP surrogate modelling is associated with unique Reproducing Kernel Hilbert Space (RKHS). Let & be symmetric positive definite kernel on Rdd and set (t) = k-(t x). For given compact space X C IRdd let Φ(X) be the space of functions with the mapping X - R, spanned by the kernel lix. ((X) is provided with the inner product represented as (kx, kxx) =
k(x x'), then the completion of (xx) is the reproducing kernel Hilbert space Hx))) of k on X.
3

---

3 Framework
We propose Human-AI Teaming (HAT) based BO framework involve the human experts in the optimisation loop to accelerate the optimisation by utilising their knowledge in the surrogate mod- elling. First, we discuss the additional knowledge accessible human experts and then we provide two variants of the HAT framework differing mainly on how the expert knowledge is communicated. The optimisation problem is same as that of the standard BO described in the Background section.
3.1 Human Experimenter Knowledge
In many real-world applications, we can expect the human experts to have knowledge about the correlations structure in the underlying function but not about the shape of the function about the location of the optima. With such knowledge, expert may be able to provide a better recommenda- tion, at least in the beginning. We use those expert suggestions at different stages of the optimisation process to assess the deviations of the current Al model and correct them accordingly. The inherent knowledge about the correlations structure of the objective function can take the form of full or par- tial knowledge of the kernel function, e-g., function is smoother along one dimension than the other. In either case, the expert can provide input regarding the relative difference between two locations in the search space in terms of their utility as the next sample point based on her cognitive model and the selection criteria. For example. if the expert knowledge of the kernel says that the objective function along one dimension is smoothly varying then an expert may be able to adjust the current recommendation by "pushing" that variable further in the direction that takes it away from the ex- isting observations. If the expert has full knowledge of the kernel then she may be able to perform such rectifications along all the input variables. We also assume that the expert running form of Bayesian optimisation strategy as it has been demonstrated that human active search typically mimies Bayesian optimisation [Borji and Itti 2013 Next, we discuss two ways of giving feedback.
3.2 Rectifying Current Recommendation
In this approach, we propose to correct the previously suggested Al model's observations 2'4 in the light of human expert knowledge. The human expert knowledge is represented using set of observations X generated using her cognitive model reflecting full or partial knowledge of the kernel function. At each iteration the kernel hyperparameters O* Rdd of the AI model (i.e., the BO process) are tuned by maximising the log- likelihooo mentioned in Eq. (I). Using this tuned GP surrogate, the Al model suggests the next potential candidate (x/+1) ) for the function evaluation. In the iterations (t. E H) that involve the expert rectification of the current Al recommendation, expert suggests the next potential candidate (xf=1) based on her cognitive model. Then, based on such previous expert recommendations we add constraints on the model selection via acquisition functions (UGP-UCB) such that the next potential candidate suggested by the AI model has to take into account the corrections enforced by the expert observations x vi E Hl collected in X Here we assume that a human expert is also using a form of BO strategy but with a better model. We should note that the expert does not necessarily need to provide the optimal point according to the correct formulation. The expert needs to only provide a more promising location than what the machine recommended. Hence, the constraints take the form acquisition function (UGF-UCB) ) at x is better than the acquisition function at x" The resulting constrained model selection problem is given as:
0' arggaax log S.L. MERAECR(x IP MCPoECR(x (x<" Vx E xx", x E E.. ie H (4)
where D corresponds to the set of all observations {(x, y = f(x)) accumulated by the AI model, x," corresponds to the 7thh observation suggested by the Al model collected in X These constraints (collected in a set C) are enforced in the model selection process in the next round. The intuition behind this constraint set is to restrict ourselves to the hyperparameter distribution that make the AI acquisition function more closely resemble the human acquisition function.
One may wonder (i) why human expert would behave like BO algorithm?, and (ii) why humans themselves cannot perform the optimisation?. To answer the former we refer to the work Borjil and Itti [213)] where they showed that when humans are presented with a search problem they often tend to behave like a BO strategy, i.e. we tend to mix a bit of exploitation and exploration when sampling for the next point. Here we assume that the strategy is like the GP-UCB strategy.

---

Algorithm HAT Rectifying Recommendations (HAT-RR)
Input: Initial observations Dr- {(x,,', Vir)), Sampling budget T
1. Initialise expert suggestions X 0, Al suggestions X' 0, Expert suggestion iterations
= 0, Constraints list C = (
2. for iterations = ,..... do
3. Optimise O- argmax log L, such that constraints in C are satisfied
if expert imputs are available i.e., X * 0, then
Add constraint UGP-UCB (x. (D-1) UCP (D/ to
V x E XA X E XX Vi E H
4.
Find the next query point x argmax (cG).
x
5.
X = x (x
6.
if human expert wants to intervene, then
7.
Obtain expert input x
8.
X - x U (x (xf+1) X,+1 =
9.
else X(+1 = x
10. Query the objective function fxx to find uc+1 = f(x(+1) + c141
Augment data D1a+1 Di.t. U ((8++1.94+1)).
12. end for
However, as we will see in the experiment, even if a human expert uses a noisy version of the GP- UCB strategy, proposed method still be robust to that. To answer the latter, say that while human may possibly get to the optima, may be extremely taxing for her to do so at every iteration, and also when the number of observations specify the optima of their acquisition function, but to suggest better location than x We believe humans we may be able to do that more easily than say running the actual BO which involves solving the global optimisation of the acquisition function. The complete procedure for Human-Al Teaming (HAT) with experts rectifying recommendations (HAT-RR) given Algorithm
high. Here, we do not need the expert to
3.3 Good Regions vs Bad Regions
In contrast to aforesaid approach, the expert can suggest a good region and a bad region to sample next. A good region is defined by set of candidate points that promise high probability of finding the optima i.e. a set of points that have similar utilities to be the next sampling location. Further, such good candidate points can be seen as the locations suggested by maximising expert's acquisition functions. On the other hand, a had region corresponds to the portion of the input space that has very low probability further improve the current solution. A bad region be visualised as a set of points that have very low utility to be the next sampling location as indicated by the low values of the acquisition function. To keep the presentation simple, we formulate our problem based on a good point and a bad point. The algorithm can be generalised to the case of region by adding constraint per pair of samples from those regions.
The expert suggestions about the good point (x²) and the bad point (x") is used to fine-tume fne-tune the AI model by replacing its current optimal hyperparameters (9*) with hyperparameter set (0+) that is more aligned with the expert's cognitive model, in addition to achieving good likelihood. To accom- plish this construct co-objective that maximises the value difference between the acquisition function of the good point and the bad point. The co-objective is mathematically represented as:
θ argmax Σ (Marucs (D,-1) (P:( starte-
Uea (x: (D-1))), s.t (log L) 2 A(log L) (5)
IE
where corresponds to the set of all observations accumulated by the model. The bi-objective problem is solved in two steps: (i) first, the pure maximum likelihood (Eq. (4) based hyperpa-
5

---

Algorithm Human-AI Teaming: Good Vs Bad Regions (HAT-DM)
Input: Initial observations Dr- {(xi,' V1 Sampling budget T, Threshold A
1. Initialise expert suggestions X 0, Human suggestion iterations H
2. for iterations ł = ...... 7 do
3.
human expert wants to intervene, then
4.
Obtain good and bad points x x from the expert
X = X (x] x H = {t}
5. Optimise hyperparameters: θ argmax log L
9
6. expert suggestions are available i.e. x 0, then
θ argmax Σ (uGR.UCB (x. (x-(P:-)) uGR-UCB (xP
such that (log L)et 2 A (log L)e.
7. Find the next query point X = argmax UGP. uGG.UCc(x))
8. Query the objective function f(x) to find yt+1 = f(x(+1) + c+21
9. Augment data D1s+1 = Di.t U {(x++1. M(+1)}
10.
end for
rameter estimation (0* argmax log L) is performed, and then (ii) the co-objective (Eq. (5)
is solved by constraining the likelihood to be A-close (threshold) to the likelihood obtained in the previous step. The overall algorithm for the HAT framework with the aforementioned difference maximisation strategy (HAT-DM) is provided in Algorithm 2
Theoretical Analysis
Our theoretical analysis relies on the notion Sobolev spaces [Tartar 2007 and its extension to Hilbert spaces. We start with the preliminary results required for the theoretical analysis of our proposed method. Chowdhhuy and Gopalan [2017] have discussed the regret bounds of variant of the GP-UCB algorithm terms of the norm bounds defined on the objective function
Lemma 1: Let f X R. where X Rd and d is the number of dimensions. Let Hy be the Reproducing Kernel Hilbert Space (RKHS) associated with the kernel function k(.,-) such that the RKHS norm is bounded in the ball of radius R i.e., ||fllk BR. Then there exists a sequence of trade-off factors Br 5) such that it holds with probability at least 1 that [f(x) [x()) S (B, E
Lemma states that ff is highly probable be contained in the confidence intervals induced by the GP predictions (-+(-) and (-)) using kernel k X x X + E with the appropriate scaling induced by 3, as discussed Chowdhury and Gopalan 2017]. As & defines the size of the function space, the RKHS norm f| = (f,, f) measures the complexity of functions f e H4. By considering larger norm-balls, we account for more complex functions in the given input space.
In the Human-AI Teaming (HAT) framework, the kernel hyperparameters θμAT are tuned in the light of additional channel of information provided by the human experts. The expert suggestions incorporated in the modelling process have a significant impact on the resultant hyperparameter distribution (see Figure Ta and [b) and thus, the latent function space. Our HAT framework disregards some portions of the hyperparameter distribution that violate the constraints added the modelling process and restricts itself to the (narrower) feasible regions. Further, the search for better hyperparameter set that more aligned with the expert's cognitive model in the vicinity of the optimal hyperparameter set obtained by maximising the likeliold also results in narrower

---

distribution (governed by the threshold A). Thus, our proposed framework deals with a narrower distribution of functions controlled by the modelling constraints enforced in the optimisation process. Such narrower distributions are spanned by the RKHS HisepNT We characterise the RKHS Haana of our framework using the Sobolev spaces [Trtarr 2007 Sobolev Hilbert space H* special case of Sobolev spaces obtained by restricting the RKHS associated with the kernel function kie. With mild regularity assumptions the boundary conditions and the smoothness of Sobolev Hilbert spaces, we draw similarities between the underlying constraints in Sobolev Hilbert spaces and the constraints enforced to construct RKHS Hkeua in the HAT framework. We use the mathematical formulations of the RKHS norm in Sobolev Hilbert spaces to characterise the RKHS norm of the HAT framework. We refer to the supplementary material for the definition of Sobolev spaces. The following theorem establishes the relation between the RKHS norms of the HAT framework and the standard Bayesian optimisation framework.
Theorem 1: Let H(RR) be the space of real continuous functions Js E (2R( Let keya be the kernel function with hyperparameters ①HAT used in the HAT framework. Let BR be the norm-ball of radius ReAT induced in HAT based BO and let BRem be the norm-ball for the standard BO. If Hkeaa (2) the reproducing Hilbert space of functions fx = gix associated with the kernel kean then with high certainty holds II kogg S BBRur and BRAs S BRam.
The proof of Theorem 1 is provided in the supplementary material. With the results established in Theorem 1, we conclude that the additional channel of human experimenter knowledge enforced as constraints results in narrower distribution for and thus significantly reducing the radius (RHAt) of the norm ball i.e., BR S. BR Now, we discuss the implications of the induced norm bounds on the overall regret of the algorithm.
In ()) where nt-1 is the associated information gain (Chowdhury and Gopalan 2017 after rounds and is the white Gaussian noise. Let ll I BR. If BR BRED then the following holds with the probubility at least 8,
Corollary 1: Pick ā E (0, Let (R, = BR + e 2( + 1 In
PIRTA RT VT 21
where Ry is the cumulative regrel given by Rr = Et= and r, is the instantaneous regret given by ri f(x") fxxx).
Proof: As discussed in Lemma 1, the RKHS norm associated with a kernel function k(;)) is bounded within the ball of radius R ie., ||fll < BR. Further, it states that with high probability lf(x) - [x)] S (x)
satisfied if 3,'s are appropriately chosen. If (x) holds, then the instantaneous regret is upper bounded by ( (see Lemma in (Srinivas et al. (2012). Therefore, the instantaneous regret r, is greatly influ-
[f(x) [xx)) S
enced by the choice of Br. By setting the value of 3r as 3, BR + 222(111 + 1+ in f) (Chowdhury and Gopalan] 2017). we can impose tighter bounds on the cumulative regret Ry. The results of Theorem state that, our proposed method places a tighter bound on the RKHS norm ball such that BREa I BR Such restrictions forces smaller values for Br in our proposed method, thereby reducing the instantaneous regret тe significantly. As a consequence, the overall cumulative regret R = Et= is also tightly bounded and grows at most as O (BR T/T + /Trr(rr + In })), where pr is the kernel associated maximum information gain Srinivas et al. 2012] after T iterations. Thus the result stated above follows.
5 Experiments
We evaluate the optimisation performance of our proposed methods on various synthetic functions and several real-world datasets. First, we validate our methods by demonstrating its sample effi- ciency in the global optimisation of synthetic benchmark functions. Next, we tune the hyperpa- rameters of the Support Vector Machine (SVM) classifier operating on publicly available real-world datasets. We compare the performance against different variations of the BO algorithm. The empir- ical results demonstrate the superiority of our proposed framework.
7

---

5.1 Emulating Human Experts
The human expert is expected to know the underlying structures of the given imput space. Here we assume that the expert is aware of the optimal ground truth kemee hyperparameters (Oar). We are motivated from the findings in Borji and Itti (2013) that the intelligent search strategy used by humans closely resembles Bayesian optimisation. Therefore, we run separate GP-UCB based BO algorithm with the ground truth kernel (ker) for emulating human experts behaviour in our proposed framework. The knowledge about the (ground truth) kernel function is obtained apriori from a large number of random samples of the objective function under consideration. Further, the human experts are given option intervene at any stage. In this work, we have assumed that human expert intervenes at every third iteration the optimisation process.
We consider the following variants of standard BO and HAT based BO in our experiments.
HAT-RR: Human-Al Teaming framework with recommendation correction as mentioned in Algorithm [ The SE kernel is used as the ground truth kernel (ker) ) for this baseline.
HAT-RR (KFO): Similar to HAT-RR, except that the ground truth kernel is chosen to be the kernel tuned from a Kernel Functional Optimisation (KFO) procedure (Venkatesh al.) 2021], a more expressive form of kernel than the Squared Exponential (SE) kernel.
HAT-DM (KFO): HAT framework with the expert feedback about good and bad regions (mentioned in Algorithm 2 and the kernel being tuned with KFO procedure.
STD-BO: A standard Bayesian optimisation algorithm mentioned in the supplementary material.
STD-BO (MOD): Another variant of standard Bayesian optimisation procedure but naively considers the suggestions provided by human experts in its observation model.
In the aforesaid methods. we have used SE kernel (Asse) for fitting GP surrogate models. The length- scale parameter of kSæ is tuned in the interval 10 1]. We standardise the function output, and thus use unit signal variance for kSE. We follow the guidelines mentioned in the Kernel Functional Op- timisation (KFO) framework (Venkatesh et al.) 2021) for the hyperparameter selection of both the SE kernel (Ksu) and hyperkernel (x) used in the KFO framework. The hyperparameters of ÆSE i.e., the lengthscale ()) and the signal variance (73) are tuned in the interval [0.1, 1]. as the bounds are always normalised. The hyperparameters dn and of the hyperkemnel x used in the KFO framework are tuned in the interval (0, 1] and (0, 1), respectively. We refer to the supplementary material for the additional details of the KFO framework used. We use 1' d 2 initial observations for d d--dimnniionl problem. The threshold (A) used in Algorithm
2 is set at 95%.
5.2 Synthetic Experiments
First, we evaluate our approach by finding the global optima of benchmark function (Oscillator 2D, see Figure Ta]. The objective function considered has varying smoothness across its input dimen- sions. The expert is assumed to be aware of this smoothness information (lengthscale) via ground truth kernel i.e,, a kernel with the optimal hyperparameter set tuned the light of numerous data points. The initial observations are chosen to be in the smoother regions of the objective function such that the methods have not much prior information about the input space. Comparing the kernel learning performances revealed that our approach encouraged larger lengthscale for 22 and shorter lengthscale for Φ1. The hyperparameter distribution along 22 is depicted in Figure 1b
Furthermore, we evaluate our proposed methods with the following multi-dimensional benchmark functions (Surjanovic and Bingham 2017] with multiple local optima: (i) Ackley 1D, (ii) Gramacy
& Lee ID, (iii) Branin 2D, (iv) Oscillator 2D, (v) Hartmann 3D, and (vi) Hartmann 6D. We compare
the performance of our methods against the baselines by plotting the simple regret (f:t) given by:
f(x") - max f(x:)
(6)
x.ED
where f(xx) is the true optima of the objective function. We plot the simple regret for x d + iterations, for given d d-dimensional problem. The convergence results obtained for the synthetic experiments after random repeated runs are shown in Figure is evident from the results that our proposed method has outperformed the baselines with KFO being better than the SE kernel.
8

---

Ackley 1D
Lee
0.5
0.8
0.5
AAT-590KF0)
0.8
0.7
NAT-DMBCTO SID-EC1M0D)
(N SID 20(000)
Simple Regres
Simple Regret
Simple Regres.
STD.60
STD
0.3-
0.4
0.2
0.3
0.2
0.2
12
14
0.0
10
12
14
3
12
35
18
21
Itarations
Oscilla tor 2D
Hartma rin 3D
Hartmann
HAT-45
Simple Regtes
0.5-
0.5-
HAT HATAR/(FF)
STD 50110D)
HAT HATONICCOO
STD 50
Simple Regree
0.4
0.5
STO (0--000000)
HAT (M6
STD-50
H51 441-RRPFF
Simple Regret
9.4
0.5
STD-60
B00MOD!
(HIKFO)
0.4-
9.3
02-
0.1
0.1
0.1
3
12 Iterations
21
0.0
Iterations
Iterations
Figure 2: Simple regret vs iterations for various benchmark functions. We plot the mean regret along with its standard error obtained. The experimental results are after 10 random repeated runs.
5.3 Real-world Experiments
We evaluate our proposed method in tuning the hyperparameters of Support Vector Machine (SVM)
classifier operating on real-world datasets. In our experiments, we consider dineniiiimnsi real- world datasets publicly available in the UCI repository (Dua and Grafi] [017]. The real-world datasets used are randomly split into training set containing 80% the total instances and a test set consisting of the remaining 20% of the total instances.
We use C-SVM with Radial Basis Function (RBF) kernel minimise the test classification error (E.). We tune the SVM hyperparameters i.e., the cost parameter and the RBF kernel width in the exponent space of [-3,3] and [-5, 1]. respectively. The minimum classification error (in %) ob- tained for the test set averaged over 10 random repeated runs are shown in Table [ We italicise the best results of our HAT-RR framework to demonstrate that our proposed method outperforms the standard Bayesian optimisation algorithm even if the ground truth kernel (squared exponential kernel) used is not very expressive of the given input space. is even better when more expressive kernel (KFO tuned kernel) is used. Further, we believe that the superior performance of HAT-RR framework against the HAT-DM framework is because HAT-RR directly incorporates the sugges- tions from the human expert for function evaluation. whereas HAT-DM assesses and corrects the deviations in the current model by maximising the distance between good and bad regions.
5.4 Robustness to Imprecision in Human Knowledge
In addition to the experiments discussed above, we further evaluate our proposed method to show its robustness in model learning against the noisy human expert inputs. In this experiment, human errors by corrupting the human inputs by adding suitable noise to the (i) kernel parame- ters, and (ii) the trade-off factor Sr. In order to add noise in the kernel parameters, we use Multi Kernel Learning (MKL) [iolli and Donini 2015) a weighted combination of SE kernel (KSE), linear kernel (kT.N) and polynomial kernel (kpor.) i.e., k(x, x') = wi kis(xx, x') + w2 KLNN x') + wa kpor. (x, x') for the ground truth kernel. We add a white Gaussian noise to the weights w i.e. wi ~ N (w,, 02xx to emulate the human expert error in understanding the ground truth knowledge. To account for the human eтror in providing suggestions, we add noise to the trade-off factor S
emulate
using a Gamma distribution (T) ie., 3, r 82 & In all our experiments, we have set the noise variance
Oē. = 0.1. We evaluate our proposed method supplemented with the imprecise human knowledge using the following synthetic functions: (i) Levy 2D, (ii) Shubert 2D, and (iii) Egg 2D. We compare the convergence results with the standard Bayesian optimisation algorithm. The results obtained for the aforesaid synthetic functions shown in Figure 3 It is evident from the results

---

Table 1: SVM classification enror rates obtained for the real-world datasets using different algo- rithms. Each cell value signifies the mean test classification eтror and its standard deviation obtained after 10 random repeated runs. Bold values indicates the best performance among all the columns. Lower the better. We italicise the best results of our HAT-RR framework to emphasise its superiority against the standard BO even when standard unexpressive (SE kernel) used.
Dataset
HAT-RR
HAT-RR (KFO)
HAT-DM (KFO)
STD-BO
STD-BO (MOD)
WDBC
0.98=0.2
0.60 0.75
0.95 + 0.45
1.5 0.5
1.24 + 0.2
lonosphere
5.15-0.4
5.25 + 0.10
6.21 + 0.81
8.9 0.2
6.02 + 0.6
Sonar
6.46=0.3
6.11 0.37
6.84 + 0.92
8.21
8.44 0.2
Heart
10.2=0.3
10.25 - 0.41
10.97 + 0.75
11.7
11.10 1.4
Seeds
2.8.10.14
2.51 + 0.65
2.63 + 0.47 0
3.4
2.58 0.8
Wine
0
0
0 14.52
Credit
12.7=1.3
12.42 0.60
12.95 0.78
1..1
Biodeg
13.4=0.2
13.88 = 0.42
14.09 = 1.83
16.8
15.05 + 0.6
Car 0.21=0.1
0.35 0.17
0.30 + 0.53
1.9
0.39 0.7
Ecoli
1.67=0.2
1.21 + 0.38
1.94 =
2.1
2.01 0.3
that our method performs on par with standard BO even when the expert knowledge is imprecise, as we try to incorporate expert knowledge to further improve the existing log-likelihood estimates.
Levy 2D
Egg 2D
0.8
HAT4R STD 60 HATON
HAT-RR STD-80
HAI-R STD-80
0.5
Simple Regret
0.6-
HAT DM
Simple Regret
0.5-
HAT ON
0.4
Simple
0.3
0.2
0.2
0.0
0.9
122
15
18
21
24
15
18
21
24
12
15
18
21
Berations
Htrrations
Rerations
(i)
(ii)
(iii)
Figure 3: Simple regret plot for (i) Levy 2D function, (ii) Shubert 2D function, and (ili) Egg 2D func- tion obtained using noisy HAT-RR algorithm, noisy HAT-DM algorithm, and standard BO (STD- BO). We plot the mean regret along with its standard error computed after 10 random repeated runs.
To further validate our proposed Human-AI AI collaborative Bayesian optimisation framework. we have conducted study with real human experts. We have provided the additional experimental results in the supplementary material (A.3). The code base used for the experiments mentioned above is available at htthtttt////gtt thub. com/ma toa www.rrunnkuaaaaannaaaa Teami ng
Conclusion
We propose novel Bayesian optimisation framework to accelerate the optimisation process by incorporating additional information available from human experts. We present two different expert intervention strategies. In the first strategy, we let the expert correct the current recommendation. In the second, we seek expert's hunch on good region vs bad region for the selection of next sample based the current observation set. We then incorporate such feedback constraints in the model selection. We theoretically analyse our framework to show that expert's knowledge can improve sample efficiency. The experimental results show that method outperforms other baselines.
Acknowledgements
This research was partially funded by Australian Government through Australian Research Council (ARC). Prof. Venkatesh is the recipient of ARC Australian Laureate Fellowship (FL170000006).
10

---

References
Aiolli and M. Donini. EasyMKL: a scalable multiple kernel learning algorithm. Neurocomputing, 169:215-224, 2015.
M. R. Andersen, Siivola, and A. Vehtari. Bayesian optimisation of unimodal functions. In Neural Information Processing Systems workshop on Bayesian optimisation, 2017.
A. Borji and L. Itti. Bayesian optimisation explains human active search. Advances in Neural Information Processing Systems, 26:55-63, 2013.
D. M. Bossens and D. Tarapore. Rapidly adapting robot swarms with swarm map-based Bayesian optimisation. In In IEEE International Conference on Robotics and Automation, pages 9848- 9854. IEEE, 2021.
E. Brochu, V. M. Cora, and N. De Freitas. A tutorial on Bayesian optimisation of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint arXiv: 1012.2599, 2010.
C. Burges. tutorial on support vector machines for pattern recognition. Data mining and knowledge discovery, 2(2):22 1998.
M. Carroll, R. Shah, M. K. Ho, T. Griffiths, S. Seshia, P. Abbeel, and A. Dragan. On the utility of learning about humans for human-Al coordination. Advances in Neural Information Processing Systems, 32:5174-5185, 2019.
S. R. Chowdhury and A. Gopalan. On kernelised multi-armed bandits. In International Conference on Machine Learning, pages 853, 2017.
A. Christmann and I. Steinwart. Support Vector Machines. 01 2008. ISBN 978-0-387-77241-7 doi: 1111077978-00337-7772222
A. De, P. Koley, N. Ganguly, and M. Gomez- Modriz-odri Regression under human assistance. In Proceedings of the AAAI Conference Artificial Intelligence, volume 34, pages 2611-2622 2020.
D. Dua and C. Graff. UCI machine learning repository, 2017. URL htttttt/aacchiee...... ics. uei. edu/ml ml [Online; accessed 21 SSpteember-2022
Frazier. A tutorial on Bayesian optimisation. arXiv preprint wwXivv1807.028111 2018.
R. Garnett, M. A. Osborne, and S. J. Roberts. Bayesian optimisation for sensor set selection. In Pro- ceedings of the Ninth ACM/IEEE International Conference on Information Processing Sensor Networks, pages 209-219, 2010.
E. Hechler, M. Oberhofer, and T. Schaeck. Limitations of AI. In Deploying Al in the Enterprise, pages 299-312. Springer, 2020.
T. T. Joy, S. Rana, S. K. Gupta, and S. Venkatesh. Flexible transfer learning framework for Bayesian optimisation. In Pacific-Asia Conference Knowledge Discovery and Data Mining, pages 102- 114, 2016.
H. J. Kushner. A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise. Journal of Basic Engineering, 86(1):97-106, 1964. ISSN 0021 0021-9223. doi: 10.1115/1.353121.
C. Li, R. Santu, S. Gupta, V. Nguyen, S. Venkatesh, A. Sutti, D. Rubin, T. Slezak, M. Height, M. Mohammed, and I. Gibson. Accelerating experimental design by incorporating experimenter hunches. In 2018 IEFE International Conference on Data Mining (ICDM), pages 257-266. 2018. doi: 10. 10.1100 109111/../20011111 00001..
M. Maadi, H. Akbarzadeh Khorshidi, and U. Aickelin. A review on human-Al interaction in ma- chine learning applications and insights for medical applications. International Journal of Envi- ronmental Research and Public Health, 18(4), 2021. ISSN 1660-4601.

---

D. Madras, T. Pitassi, and R. Zemel. Predict responsibly: improving faireess and accuracy by learning to defer. arXiv preprint arXiv:111.00664, 2017.
R. Marchant and F. Ramos. Bayesian optimisation for intelligent environmental monitoring. In IEEE/RSJ International Conference Intelligent Robots and Systems, pages 2242-2249, 2012.
R. Martinez-Cantin, N. de Freitas, A. Doucet, and J. A. Castellanos. Active policy learning for robot planning and exploration under uncertainty. Robotics: science and systems, volume 3, pages 321-328, 2007.
H. Mozannar and D. Sontag. Consistent estimators for learning defer to an expert. In International Conference on Machine Learning, pages 7076-7087, 2020.
J. Rihhimäki and A. Vehtari. Gaussian processes with monotonicity information. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 645-652, 2010.
M. Roccetti, G. Delnevo, L. Casini, and P. Salomoni. A Cautionary Tale for machine learning design: why we still need numan-assistedd big data analysis. Mobile Networks & Applicctiins, 25 (3), 2020.
A. Shah, A. Wilson, and Z. Ghahramani. Student-t processes alternatives Gaussian processes. In Artificial intelligence and statistics, pages 877-885, 2014.
B. Shahriari, K. Swersky, Z. Wang, R. P. Adams, and N. De Freitas. Taking the human out of the loop: a review of Bayesian optimisation. Proceedings of the IEEE, 104(1):148 175, 2015.
A. Shilton, S. Gupta, S. Rana, and S. Venkatesh. Regret bounds for transfer learning in Bayesian optimisation. In Artificial Intelligence and Statistics, pages 307-315, 2017.
N. Srinivas, A. Krause, S. M. Kakade, and M. W. Seeger. Information theooretich regret bounds for Gaussian process optimisation in the bandit setting. IEEE Transactions on Information Theory, (885):3250-2265.22 2012.
S. Surjanovic and D. Bingham. Virtual library of simulation experiments: Test Functions and Datasets, 2017. URL WRLhhttp://wwer sfu. ca/ rsse/sssujan [Online; accessed 21 21-September- 2022].
L. Tartar, An introduction to Sobolev spaces and interpolation spaces, volume 3. Springer Science & Business Media, 2007.
A. K. A. Venkatesh, S. Rana, Li, S. Gupta, A. Shilton, and S. Venkatesh. Bayesian optimisation for objective functions with varying smoothness. In Australasian Joint Conference on Artificial Intelligence, pages 460-472, 2019.
A. K. A. Venkatesh, A. Shilton, S. Rana, S. Gupta, and S. Venkatesh. Kernel functional optimisation. In Advances in Neural Information Processing Systems, 2021.
Z. Wang, B. Shakibi, L. Jin, and N. Freitas. Bayesian multi-scale optimistic optimisation. Artifi- cial Intelligence and Statistics, pages 1005-1014, 2014.
B. Wilder, E. Horvitz, and Kamar. Learning to complement humans. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, pages 1526-1533. Inter- national Joint Conferences on Artificial Intelligence Organisation, 2020.
C. K. Williams and C. E. Rasmussen. Gaussian processes for machine learning, volume 2. MIT press Cambridge, MA, 2006.
J. Wilson, F. Hutter, and M. Deisenroth. Maximizing acquisition functions for Bayesian optimisa- tion. In Advances in Neural Information Processing Systems, pages 9884-9895, 2018.
T. Zhang, Q. Zhao, K. Shin, and Y. Nakamoto. Bayesian-opiimisstion--oo based peak searching algo- rithm for clustering in wireless sensor networks. Journal of Sensor and Actuator Networks, 7:2, 2018.
12

---

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [Yes]
(b) Did you include complete proofs of all theoretical results? [Yes]
3. If you experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either the supplemental material or as URL)? [Yes]
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e-g., code, data, models) or curating/releasing new assets...
(a) If your work existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]
(d) Did you discuss whether and how consent was obtained from people whose data you'le re using/curating? [N/A]
(e) Did you discuss whether the data you are using/curating contains personally identifi able information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]
13


Reviews:
[{"review_id": "ulbAFVkJOt", "reviewer": "Reviewer_R7wD", "summary": "This paper proposes a framework that makes use of expert's knowledge in improving the effectiveness of Bayesian optimization. The Human-AI Teaming (HAT) based Bayesian optimization framework takes expert inputs as constraints via the following two ways:\n- Rectifying recommendations by making use of the correlation structure of the experimental system by updating the hyperparameters with the constraints that the acquisition function of the human model is better than the one of the AI model. \n- Specifying good and bad regions based on the current observation set by updating the hyperparameters that optimize the difference between the acquisition function of the good regions and the bad regions and achieve at least \\Delta of the original likelihood. \nFurthermore, the paper also provides theoretical analysis that shows additional information from human experts enforced as constraints result in a narrower distribution, thus improving sample efficiency. \n\nFinally, the effectiveness of the proposed framework is tested using both synthetic data and real-world experiment of tuning the hyperparameters of SVM. ", "questions": "In the synthetic experiment section, Figure 2 shows the simple regret for 10*d+5 iterations for the different models. Why choose \"10\" here?  It seems that after certain iterations, although the human AI teamed method converge faster (i.e., big drops in simple regret in early iterations than standard BO procedure), they reach the optimal simple regret almost at the same iteration as the standard BO procedure. For example, Ackley 1D, human AI teamed model reached 0.0 regret around iteration 13 and standard BO around iteration 14. Since the iteration is capped at \"10*d+5\", it's not clear if human-AI will get to the global optimal faster than the standard BO procedure. The experiments does show that when the number of iterations (e.g. computation budget) is limited, using a human-AI teamed BO method will give better results.\n\nTypos: \n- Line 278: \"Algorithm 2\" should be \"Algorithm 1\" ", "limitations": "The proposed method will improve the performance of Bayesian optimization method when the computation budget is limited; however, it does not seem to find the optimal solution in significant fewer iterations. One improvement to add in the experiment section is to provide a figure of the number of iterations of the optimal regret (after the optimizer is converged) for all the comparing models. ", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength: \n- This paper is well written and flows well. \n- This paper provides theoretical analysis of the proposed method. \n- This paper provides an original idea of incorporating human experts knowledge into the Bayesian optimization procedure through constraitns\n- The effectiveness of the proposed method is demonstrated in both synthetic and real-world experiments \n\nWeakness: \n- It's not clear whether the proposed method will achieve the optimal solution faster than the standard Bayesian optimization based on the presented results. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "T-W94xjjA5h", "reviewer": "Reviewer_dMaE", "summary": "The paper proposes a new method for human-AI collaboration in the context of Bayesian optimization, where the optimization algorithm can get occasionally help from a human expert with deeper knowledge about the underlying physical phenomenon, either in the form of a correction of the current recommendation within the search space or specifying good and bad regions within the search space. Authors provide a theoretical framework justifying their approach and an empirical validation of the method against a baseline method without expert feedback.\n", "questions": "- Could you better explain the statement at line 29, expanding more on why the nature of application ensure that there would be an expert running the optimization process?\n- Are there other works in the literature introducing human feedback into Bayesian optimization or optimization processes? I would expand the background section by mentioning the state of the art of works including the human in the loop of optimization processes. \n- How would authors explain the better performance of HAT with recommendation correction than with bad/good regions feedback? (see Fig.2)\n- In real-world experiments: which was the classification task? And, how was the feedback from humans provided in this case?", "limitations": "Authors don't address limitations of their work. \nThe main limitation of including a feedback expert in the optimization process is that the performance relies heavily on the human true level of \"experience\" about the process under analysis, and besides such feedback goodness varies from expert and expert. I suggest authors to mention their view about this and other limitations as well as improvement perspectives, in the Conclusion section.", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper contribution is clearly presented and its effectiveness in accelerating the optimization process is supported theoretically and validated both using synthetic and real world data. The  paper is articulated in a clear way and the overall quality is fair. \nThe background section and results discussion do not report other works about collaborative human-AI in optimization processes, therefore an assessment of the paper's contribution originality and significance in the context of related work is hard. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "KtWGzYZcW-8", "reviewer": "Reviewer_7Niz", "summary": "This paper proposes to augment Bayesian optimization (Gaussian processes based) formulation with human experts intervention on suggesting sample points (different from the Bayesian optimization suggestion) and human's good and bad sample regions (to be exact they are sample points instead of regions as it is formulated in the paper now).  The introduction of human intervention into BO is an interesting and potentially useful contribution. However, the nature of human intervention is not rigorous enough missing  formal  characterization of the exact requirement on human experts and practical verification of such human properties in human study. For example, we might need verifiable formal notion of what if human expert is wrong, what kind of wrongness, and how wrong the system can endure a human expert but still guarantee a bound of the optimality as well as the sampling process efficiency.     \n", "questions": "- what if human expert is wrong?\n- what kind of wrongness, and how wrong the system can endure a human expert but still guarantee a bound of the optimality as well as the sampling process efficiency?", "limitations": "Please improve the study with (1) a formal  characterization of the exact requirement on human experts and practical verification of such human properties in human study; (2) non-Gaussian processes BO formulation such as: Garnelo, Marta, et al. \"Neural processes.\" arXiv preprint arXiv:1807.01622 (2018), and its follow-up work.\n", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strength\n\t- The introduction of human expert recommended samples as well as good and bad region suggestions into the BO formulation\n\nWeakness\n\t- Experiments lack real human experts human study\n\t- No formal treatment of human experts properties and how these properties will be factored into the proposed augmented BO\n\t- The BO formalization will be better off if it can also handle non-Gaussian processes BO formulation such as: Garnelo, Marta, et al. \"Neural processes.\" arXiv preprint arXiv:1807.01622 (2018), and its follow-up work.\n\t\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept