Bayesian Active Learning with Fully Bayesian Gaussian Processes
Authors: Christoffer Riis
Francisco Antunes
Frederik Boe Hüttel
Carlos Lima Azevedo
Francisco C. Pereira
Venue: NeurIPS 2022 Accept (2022)

Abstract: The bias-variance trade-off is a well-known problem in machine learning that only gets more pronounced the less available data there is. In active learning, where labeled data is scarce or difficult to obtain, neglecting this trade-off can cause inefficient and non-optimal querying, leading to unnecessary data labeling. In this paper, we focus on active learning with Gaussian Processes (GPs). For the GP, the bias-variance trade-off is made by optimization of the two hyperparameters: the length scale and noise-term. Considering that the optimal mode of the joint posterior of the hyperparameters is equivalent to the optimal bias-variance trade-off, we approximate this joint posterior and utilize it to design two new acquisition functions. The first one is a Bayesian variant of Query-by-Committee (B-QBC), and the second is an extension that explicitly minimizes the predictive variance through a Query by Mixture of Gaussian Processes (QB-MGP) formulation. Across six simulators, we empirically show that B-QBC, on average, achieves the best marginal likelihood, whereas QB-MGP achieves the best predictive performance. We show that incorporating the bias-variance trade-off in the acquisition functions mitigates unnecessary and expensive data labeling.

Bayesian Active Learning with Fully Bayesian Gaussian Processes
Christoffer Riis
Frederik Boe Hüttel Carlos Lima Azevedo Francisco Câmara Pereira
Francisco Antunes
DTU Management
Machine Learning for Smart Mobility Sehrrii,franae fboby, cl imaz,, aamara)@dtu.... dk
Abstract
The bias-variance trade-off is well-known problem in machine learning that only gets more pronounced the less available data there is. In active learning, where labeled data is scarce or difficult to obtain, neglecting this trade-off can cause inefficient and non-optimal querying, leading unnecessary data labeling. In this paper, focus active learning with Gaussian Processes (GPs). For the GP, the birs-variance trade-off made by optimization the two hyperparameters: the length scale and noise-term. Considering that the optimal mode of the joint posterior of the hyperparameters is equivalent to the optimal bias-variance trade-off, we approximate this joint posterior and utilize to design two new acquisition functions. The first is Bayesian variant of Beery-by-Comminne the second is extension that explicitly minimizes the predictive variance through Query by Mixture of Gaussian Processcs (QB-MGP) formulation. Across six sim- ulators, we empirically show that B B-QBC, on average, achieves the best marginal likelihood, whereas QB-MGP achieves the best predictive performance. We show that incorporating the bias-vat vlas-variance trade-off the acquisition functions mitigates unnecessary and expensive data labeling.
(B-QBC), and
Introduction
Gaussian Processes (GPs) are well-known for their ability to deal with small to medium-size data sets by balancing model complexity and regularization [Williams and Rasmussen, 2006]. Together with their inherent ability model uncertaintics, this has made GPs the go-to models to use for Bayesian optimization and metamodeling [Snoek et al., 2012, Gramacy, 2020]. For both cases, the data is often scarce, making the modeling task balance between complexity and regularization, i.e., preventing severe overfitting while maintaining the ability fit nonlinear functions. Likewise, the ability quantify the uncertainty often guides the acquisition functions of Bayesian optimization and active learning schemes, which are inevitably required to build metamodels efficiently.
On the other hand, it is not flawless to use GPs in Bayesian optimization and active learning. In both cases, the same GP used in an itcrative process firstly predict the mean and variance of new data point and then secondly to use those estimates to guide the data acquisition. Since this is an iterative process, poor predictions will lead poor data acquisition, and vice versa. The problem is less pronounced for larger data sets as predictions become increasingly accurate as more data is available. However, in Bayesian optimization and active learning, where the data sets tend to be relatively small, wrong predictions can result in misguidance, thus hindering performance and efficiency. this paper, we mitigate this problem by applying fully Bayesian approach the GPs and formulating two new acquisition functions for active learning. Where a single GP trained with maximum likelihood estimate only represents one model hypothesis, Fully Bayesian Gaussian Process (FBGP) represents multiple model hypotheses at once. We will utilize this extra information
36th Conference on Neural Information Processing Systems (NeurlPS 2022).

---

to create an acquisition function that simultaneously seeks the best model hypothesis and minimize the prediction error of the GP.
The hyperparameters of a GP are typically fitted through evaluation of the marginal likelihood, which automatically incorporates trade-off between complexity and regularization [Williams and Rasmussen, 2006], also known as the bias-variance trade-off [Bishop, 2006]. However, when the data is scarce, it is more challenging to choose the appropriate trade-off, and different configurations of the hyperparameters of the GP give rise to distinct fits. We highlight this issue in Figure 1, where two seemingly reasonable fits describe the data very distinctly, which would result different. acquisitions of new data.
High noise regime
Low noise regime
Length scale
MW
0.0
Noise
Figure I: Two GPs with different hyperpara- meters. The left plot shows a GP with high noise and long length scale, and the right plot shows GP with low noise and a short length scale.
Figure 2: Joint posterior of the two hyperpara- meters length scale and noise. The posterior is multimodal with low noise, high signal, and high noise, low signal mode.
The bias-variance trade-off for Gaussian Processes For a GP with a common stationary co- variance function with a length scale and noise-term, e.g., the radial-basis function the Matérn class of functions with a Gaussian likelihood, the challenge of choosing the best trade-off between bias-variance trade-off, can be formulated using the hyperparameters of the GP. The two central hyperparameters are the length scale É and the variance of the noise of. The length scale describes how much the underlying model f fluctuates. i.e., if the length scale short or long, the model varies quickly or slowly, respectively. In terms tuning the hyperparameters, often a short length scale goes well with small noise-term since the greater flexibility of short length scale means that the noise level can be reduced. This results Conversely, a long length scale tends to increase the noise level, resulting in a rigid model with low variance and high bias [Bishop, 2006]. In the extremes, the former corresponds to white-noise process model. and the latter corresponds to a constant with white noise [Williams and Rasmussen, 2006]. Thus, there is a direct relation between the bias-variance trade-off and the values of the two hyperparameters.
flexible model with high variance and low bias.
How to choose the best bias-variance trade-off? Finding good bias-variance trade-off is non-trivial problem. In the case of small to medium-sized data sets, the joint posterior distribution of the two hyperparameters
likely to be characterized by two modes, as illustrated in Figure for the data in Figure 1. The two modes illustrate two different virianar trade-offs, which both describe the data well. However, depending on choice of trade-off, the data acquisition can be very distinct, and thus a wrong choice of mode will imply non-optimal guidance from the acquisition function.
Though the multimodal posterior has been studied for GPs before [Yao et al., 2022], the literature often scarches for the single best mode with clever initializations of the hyperparameters [Williams and Rasmussen, 1995] or by favoring small E and o² by either always initializing hyperparameters in the low noise regime or by applying strong priors [Gramacy, 2020]. However, none of these approaches directly address the core problem: which mode choose? When working with Bayesian optimization and active learning, this should ideally answered with prior information about the problem, although typically that is not available, making these approaches less practical [Antunes et al., 2018, Gramacy, 2020, Riis et al., 2021].
Our contribution We follow general approach to the problem and assume prior knowledge about the functional form of the data, kernel, nor hyperparameters. If it known that the data has a periodic trend or high noise, it is advantageous to incorporate that into the kernel and the hyperparameters by using periodic kernel and prior the hyperparameters that favor high noise, respectively. However, with no prior knowledge, we tend to use the
Radial-

---

Basis function (RBF) kernel with non-informative priors on the hyperparameters. Further, the fitted hyperparameters are often given by point estimates (e.g. fitted with maximum likelihood estimation or maximum a posteriori), but we consider multiple model hypotheses by replacing the fitting procedure of the marginal likelihood with Markov Chain Monte Carlo (MCMC) sampling to gel the joint posterior of the hyperparameters. The result is that we have multiple models (same kernel, but different hyperparameters), which represent different model hypotheses. We utilize the extra information from the hyperparameters' joint posterior to handle the bias- trade-off by incorporating the extra information into two new acquisition functions.
Our main contribution is the proposal of two new acquisition functionn for active learning that utilize the extra information from the hyperparamters' posterior estimated by MCMC seek the most reasonable mode alongside minimizing the predictive variance. Through empirical results, we show that the two acquisition functions are more accurate and robust than other common functions across multiple benchmark simulators used in the literature.
2 Related work
In this section, we review related work to the proposed acquisition functions. We cover active learning schemes for regression lasks, including Bnter-byy-oomminn and GP Gaussian Mixture Models.
Active Learning The main idea of active learning is to actively choose new data point to label and add to the current training data set, to iteratively improve the performance of the predictive model [Settles. 2009]. In the context of metamodeling or surrogate modeling of simulators, new data is often added sequentially, i.e., one data point a time [Gramacy. 2020], but in other upplications, it can be beneficial to query batches of data instead [Kirsch al., 2019].
The acquisition functions can be divided into model-based and model-free functions, where the former utilize information from the model and is often based on uncertainty measures (recently also function values and gradients [Fernandez et al., 2020, Svendsen el al., 20201), whereas the latter do not use information from the model and typically based on distance metries in the input space [O" 'NNlll et al., 2017]. Both types of functions seek to minimize the expected predictive loss of the model. Another distinction between the acquisition functions is decision-bused and information theory-bused [Houlsby et al., 201 Decision- based functions seek to minimize the expected predictive loss in the hope of maximizing the performance on the test set. Information- throonnttthrrri--bbaaneaa functions instead try to reduce the number of possible models, e.g., through the KL--ivergence or Shannon entropy.
It is not straightforward to use cntrmaation-hhooreeiii acquisition functions. However, if one has access to the posterior of the parameters, Houlsby et al. [2011] 1] have derived the algorithm Bayesian Active Learning by Disagreement (BALD), which can be applied in general. Generally, BALD seeks the data point that maximizes the decrease in the expected posterior entropy of the parameters.
(QBC) is specific acquisition function that was originally proposed for classification tasks [Seung al., 1992]. It aims to maximize the disagreement among the committee to get the highest information gain and minimize the version space, which is the set of model hypotheses aligned with the training data. The construction of the committee is the core component of QBC since is the committee's 's ability to accurately and diversely represent the version space that gives rise to informative disagreement criteria [Settles, 2009].
Query-by-Commicde
The
Query-by-Car -oymmmmteeeee can also be applied for regression problems. Krogh and Vedelsby [1995] construct the members of the commiltee by random initializations of the weights in the neural networks. RayChaudhuri and Hamey [1995] apply bagging and train the members on different subsets of the data set. In general, QBC constructed by bagging has been used as a benchmark with mixed results [Cai et al., 2013, Wu, 2018 Wu et al., 2019]. Burbidge et al. [2007] show that the less noise there is in the output, the better QBC compared to random querying. They also highlight the fact that with a misspecified model, QBC might perform worse than random querying. None of these approaches explore the usage of MCMC samples of the posterior to construct committee.
Gaussian Mixture Model Mixture models have recently been applied in active learning for classification tasks. Iswanto [2021] proposes to use Gaussian Mixture Models (GMMs) with active learning, where he designs a specific acquisition function that queries the data point that maximizes the expected likelihood of the model. Zhao et al. [2020] use mixture of GPs in active learning, where each component is fitted to a subset of the training set. The combination of
Gaussian Process

---

GMMs and GPs have previously been explored for static data sets. Chen and Ren [2009] investigate regression tasks and apply bagging, where they repeatedly randomly sample data points from the training set construct new subsets to get GPs fitted to different data.
3 Gaussian Processes
The Gaussian Processes (GPs) are the central models in this work. In this section, we give brief overview GPs before covering the Fully Bayesian GPs. For a thorough description GPs, we refer to Williams and Rasmussen [2006].
A Gaussian Process (GP) is a stochastic function fully defined by a mean function m(-) and a covariance function (often called kernel) k(:--). Given the data D = (X.y) = (mt, y6 h where ys is the corrupted observations of some latent function values with Gaussian noise l.e., yi = = + Exs E; (0, GP typically denoted as ( (x, 2²)). is common practice to set the mean function equal to the zero-value vector and thus, the GP is fully determined by the kernel (x,, m²). For short, we will denote the kernel Ko, which explicitly states that the kernel is parameterized with some hyperparameters 0. The generative model the GP can be found in Appendix A.1. Given the hyperparameters 0, the predictive posterior for unknown test inputs X* is given by pf 00 y. X, X*) = N( ")) with
1 = K2 (Ke + -2 and >' K2 - K2 (Ka + = K2
(I)
where Kg* denotes the covariance matrix between the test inputs, and Kÿ denotes the covariance matrix between test inputs and training inputs.
We use the canonical kernel automatic relevance determination (ARD) Radial-basis function (RBF) given by k (az x²) = exp (-1|æ æ'| 22|12/2e²) where is a vector of length scales f,...., Ea. one for each input dimension. Often the kernel is scaled by output variance but here fix solely focus on two other hyperparameters: length scale and noise-term. The noise-term o² is integrated into the kernel with an indicator variable by adding the term of (a==') to the current kernel [Williams and Rasmussen, 2006, Bishop, 2006].
and
Fully Bayesian Gaussian Processes (FBGP) An FBGP extends GP by putting prior over the hyperparameters p(0) and approximating their full posteriors. The joint posterior then given by
p(f, 0ly. X) p(u))))((110xX))000
and the predictive posterior for the test inputs X is
pvy" = /f 10 (0)00
(3)
where the conditioning X and X have been omitted for brevity. The inner integral reduces the predictive posterior given by a normal GP, whereas the outer integral remains intractable and is approximated with MCMC inference with M samples as
M Σ M
p (3)1))) - / (y 1) p(0)))
ply'ly.
0, ~ p(0
(4)
Adapting the hyperparameters of an FBGP is computationally expensive compared to the approach with GPs and maximum likelihood estimation. However, in Bayesian optimization and active learning, the computational burden for querying new data point will often be of magnitudes higher. For example simulators, the computational cost of querying new data point is, in general, expensive and can take minutes and hours [Gorissen et al., 2009, Riis et al., 2021, Chabanet et al., 2021].
4 Active Learning
In this section, we lay out the most common acquisition functions and then propose first a Bayesian variant of Boury-by-Commee and second extension motivaled by Gaussian Mixture Models, which seek to minimize both the predictive variance and the number of model hy hypotheses.

---

Many active learning acquisition functions are based on the model's uncertainty and entropy and can thus be denoted as Bayesian active learning acquisition functions [Seules, 2009, Gramacy, 2020]. The most common acquisition function is based the predictive entropy and denoted Active Learning MacKay (ALM) [MacKay, 1992]. All the following objective functions query a new data point by maximizing the argument æ. In the following, we write a new test point x" as æ for brevity. All the acquisition functions choose a data point among the possible data points in the unlabeled pool U.
Entropy (ALM) For Gaussian distribution, the Shannon entropy H[: is proportional to the predictive variance (1x) (derived in A.2), so ALM is given as
ALM H [y|a, DJ (xx)
Intuitively, ALM queries the data point, where the uncertainly of the prediction is the highest.
If we have access to the posterior of the model's 's hyperparameters, we can utilize acquisition functions with an extra Bayesian level. The posterior of the hyperparameters of an FBGP can be estimated with MCMC such that GPs with different kernel parameters can be drawn, e.g., the length scale and the noise-term l, of ~ p)) The following four acquisition functions all utilize this information and are approximated using the samples from the MCMC, cf. (4).
Entropy (B-ALM) With the information from the posterior pDO)D), the criteria for the extra Bayesian variant of ALM (B-ALM) is then given as
B-ALM = H [ ( = H EL(oTD) plyl,, 0)] x Ep(ord) 0
(6)
Bayesian Active learning by Disagreement (BALD) Another common objective in Bayesian active learning, is to maximize the expected decrease in posterior entropy [Guestrin et al., 2005, Houlsby al., 2012]. Houlsby et al. [201 rewrite the objective from computing entropies in the parameter space to the output space by observing that it is equivalent to maximizing the conditional mutual information between model's parameters θ and output I I(0, y/æ, D). The acquisition function is denoted Bayesian Active Learning by Disagreement (BALD) and the criteria is given by:
10, ylar, D = H[ylæ, D) )) [y| Ø]]
(7)
BALD was originally derived for non- parametric discriminative models but has recently been extended to batches and deep learning with great success [Kirsch et al., 2019]. In the context of non-pan paraamametti models, such as GPs, the models parameters now correspond the latent function f. For a regular GP, BALD is equivalent to ALM (cf. A.3), although that is not the case for an FBGP. If we let the hyperparameters be the main parameters of interest and set f to be nuisance parameter, BALD can be writlen as (cf. A.3.1):
BALD = H [Epoor))y/ D, 0] - Ep(or [H[y| 0]]
(8)
Bayesian Query-by-Commn Committee (B-QBC) Motivated by finding the optimal birs-variance trade- off, we propose Bayesian version of the Query-hy-Commitee using the MCMC samples of the hyperparameters' joint posterior. We have previously argued that the optimal bias-variance trade-off is equivalent to the optimal mode of the multimodal posterior of the hyperparameters, which is exactly what we utilize here. We use the joint posterior of the hyperparameters obtained through MCMC draw multiple models and then query new data point where the mean predictions μe (x) of these models disagree the most, i.c. querying the data point that maximizes the variance of μe (x). Each mean predictor (*(()) drawn from the posterior is equivalent to a single model, and thus this criteria can be seen Bayesian variant a Query-by-Cone by-oomiitte, and thus denoted Bayesian Query-by wae-bbyy--oomeee (B- (B-QBC). Given that f(z) is the average mean function, B B-QBC is given as
B-QBC (uo)
Ep(orD) ((oo)))(((x()) (1)()))))() ((0)
Since the models are drawn from the hyperparameters' posterior, the collection of models is dominated by models near the posterior modes. High variance in ( thus corresponds to high disugreement between modes. Querying the data point that maximizes this disagreement, gives information about which mode is most likely to be the optimal one, and thus this can be seen as a mode-seeking Bayesian Query-b--oomiile To the best of our knowledge, are the first to propose QBC based on model hypotheses drawn from the hyperparameters' joint posterior.

---

Table Stochastic simulators used in the experiments.
Simulator
d
Noise
Input space
Previously used in
Gramacy ld
0.1
[0.5, 2.5]
Gramacy and Lee [2012]
Higdon
0.1
[0]
Gramacy and Loc [2009]. Gramacy [2020]
Gramacy2d
2
0.05
[ 6
Gramacy and Lee [2009. 2012], Sauer et al. [2022]
Branin
2
11.32
[-5, 10] x U0 15]
Keane et al. [2008]. Picheny et al. [2013). Cole et al. [2022]
Ishigami
0.187
TT,,
Marrel et al. [2009]. Cole ct al. [2022]
Hartmann
0.0192
[0. 1
Picheny et al. [2013]. Cole et al. [2022]
Gramacy1d
Higdon
Gramacy2d
Branin
www
Figure Visualization the simulators (excl. Ishigami and Hartmann due to the dimensionality).
Query by Mixture of Gaussian Processes (QB-MGP) Bayesian Query-by- by-bom---mmtte (B-QBC) seek the optimal mode, but does not take the predictive performance of model into account. Since the predictive performance, and thereby the predictive uncertainty, is also important, we extend B B-QBC consider the predictive entropy as well. We denote the new acquisition function Query by Mixture of Gaussian Processes (QB-MGP), because of its relation to Gaussian Mixture Models (GMMs). Using the MCMC samples, each prediction of the FBGP can be seen as an MGP, yielding the predictive posterior given as in equation (4): 7 Σ j== (y" (y. 02)- This hierarchical predictive posterior is a mixture of M Gaussians with mean μGM and variance C:MM defined as (cf. A.4);
M
1
(GMM((2
no, (ar)
(10)
M
j=1
M
M M Σ (ne, (x) PGMM (x))²
CCMM (T)
= M
0 (x) +
(11)
Finding the data point that maximizes the variance of the Mixture of Gaussian Processes (MGP) is now equivalent to simultancously considering the B-ALM and B-QBC, i.c., the sum of the two:
QB-MGP (c) + ()p
- (=)MM(=)))
(12)
Instead of using bagging, construct the multiple GPs by using the MCMC samples of the hyperparameters' joint posterior, and then obtain natural weighting of the GPs: the MGP will consist of more GPs with hyperparameters close to the modes than hyperparameters far away. To the best of our knowledge, are the first to use MGP in this manner for active learning.
5 Experiments
In this section, we benchmark the performance of the two proposed acquisition functions against the standard acquisition functions based on the entropy, i.e., ALM, B-ALM, and BALD, on various classic simulators used in recent literature on GPs and active learning. They are all listed in Table 1, and those with less than three inputs are shown in Figure 3. The multimodal posteriors of the FBGPs fitted to the simulators can be found in appendix A.5.
Experimental settings In the experiments, each iteration of the active learning loop, the inputs are rescaled to the unit cube [0, 1]d, and the outputs are standardized to have zero mean and unit variance. Following Lalchand and Rasmussen [2020], we give all the hyperparameters relatively uninformative /N 3) priors in log space. The initial data sets consist of three data points chosen by maximin Latin Hypereube Sampling [Pedregosa et al., 2011 I], and in each iteration, one data point is queried. The unlabeled pool U/ consists of the
use zero-mean GP with an ARD RBF kernel. In
'All of them can found at httpp://wwrt www.ww....cowwwww......www.w..ww "ed"esurjaao/ [Surjanovic and Bingham, 2022].

---

input space discretized into 100 equidistant points along each dimension. U contains more than 10, 000 data points, we randomly sample a subset of 10, 000 data points in each iteration and use that as the new pool. The inference in FBGP is carried out using NUTS [Hoffman and Gelman, 2014] in Pyro [Bingham et al., 2019] with five chains and 500 samples, including warm-up period with 200 samples. The remaining 1500 samples are all used for the acquisition functions. For all the predictions, use the best mode of the hyperparameters' posterior, since the mean is of limited value when the posterior multimodal. The best mode is computed by using a kernel density estimation with a Gaussian kernel [Pedregosa el al., 2011] The models are implemented in GPyTorch [Gardner et al., 2018]. All experiments simulators and five acquisition functions. this gives 350 active learning runs, each with a running time on approximately one hour, using five CPU cores on Threadripper 3960X. The code for reproducing the experiments is available on GitHub.
repeated ten times with different initial data sets. With seven
Evaluation It is common to evaluate the performance of active learning by visually inspecting the learning or loss curves, or by measuring the performance after a specific iteration [Gramacy, 2020, Seules, 2009]. However, both procedures are inadequate to quantify how better an acquisition function is than another if we are not interested in the performance of specific iteration but the performance in general. We are unaware of any metric for quantifying the overall performance within the regression setting, which is comparable actoss different data sets. Within the classification setting, Yang and Loog [2018] propose
the area under the learning curve (AUC) based accuracy. Since the accuracy bounded between 0 and 1. they can compare this measure across different data sets. Likewise, O' Neill et al. [2017] compute the AUC using the rool mean square error (RMSE) as the performance metric. However, since the magnitude of the RMSE is data-specific, they can not compare the performance across data sets. To resolve this problem, we suggest using the relative decrease in AUC.
We compare the relative decrease with respect to the baseline acquisition function ALM since the latter is widely used and regarded as the standard within active learning with GPs. For metric that is lower bounded by zero, such as RMSE, the AUC will give the overall error of the acquisition function. We can directly calculate the relative decrease in error from the AUCs of the active learning acquisition functions and the baseline. If the metric has lower bound, such as the negative log marginal likelihood (NLML), the interpretation less intuitive. Therefore, we make lower bound for the metric using the lowest NLML obtained across all the acquisition functions, such that the relative decrease in the AUC then can be interpreted in the same way for the RMSE. We compare all runs of each acquisition function with the 10 runs for the baseline to get a precise estimate of both the mean and standard deviation of the relative decreases. Since the relative decrease is ratio, compute the unbiased estimates using the formulas from Van Kempen and Van Vliet [2000]. See Appendix A.6 for the formulas and the pseudo-code.
5.1 Experiments with 6 simulators
We benchmark the acquisition functions on the six classic simulators. We divide the simulators into subgroups and describe how the functions work in different complexities and with multiple inputs, effectively encompassing three distinct modeling scenarios. The following paragraphs describe the active learning curves in Figure 4.
Noise or signal The simulator GramacyId has previously been used to study the effect of the noisc-term in GPs [Gramacy and Lec, 2012]. The simulator has periodic signal that is hard to reveal if the data points are not queried cleverly. Both B-QBC and QB-MGP reach convergence simultaneously, while the other acquisition functions struggle in distinguishing noise from the signal.
Linear and non-linear output regions The two simulators Higdon and Gramacy2d have been used to illustrate cases where GPs struggle in modeling the data due to the output signal having these simulators
both lincar and non-lincar regions [Gramacy and Lee, 2009]. Our experiments show the performance of the acquisition functions when the GP is an non-optimal choice of model. Qucrying data points in the lincar and non-ling pon-linear regions will yield a GP with a longer and shorter length scale, respectively. For Higdon, both B B-QBC and QB-MGP balance the sampling since the corresponding NLML and RMSE are low, likewise, for Gramacy2d, QB-MGP is achieving both the
hhttpp///sttaal.ccc//orri/ai/cie fugg

---

Gramacyld
Higdon
0.5
RAASEE
RMACE
0.00
-0.
-1.0
0.28
0.9
-.6
0.00
75
Gramacy2d
Branin
1.0
0.10
NLAL
0.08
NUHL
RMAGE
2-
-0.5
10
20
60
10
20
30
10
20
30
10
20
40
Ishigami
Hartmann
20
15
0.0
NLML
RANSE
10
RISSE
0.4
0.3
R-ALM
B-QBC
QR-MGP
Figure 4: Performance across the 10 runs 1 standard deviation. The a-axis represents the number of iterations.
Table 2: The relative decrease in the area under the active learning curves compared to the active learning curve of the baseline acquisition function ALM.
GRAMACYID
HIDDON
GRAMACY2D
BRANIN
ISHIGAII
HARTMANN
OVERALL PERFORMANCE
DIMENSIONS
2
MEAN
MEDIAN
RELATIVE DECREASE
(%6)
ALM
0.0 20.6
0.0 112
0.0 16.4
0.0
0.0 111
0.0 0.2.2
0.0 10.0
0.0
B-ALM
-1.7 20.5
111
432 13,1
21
7. 11.
48.2 1104
16.1 121.3
4.6 4.61221.3
4:2511
B-QBC
36.3 110
41.5 11.1
13.9 14.6
33, 112
37.5 10.8
75.5 15.1
41.3 116.2
36,9 16.2
QB-MGP
20.6 =0.9
15.9 +14
306 +4.
7.1 =1
21.7 +0.7
S0.1 =39
29.3 +-33.8
21.1 +238
RMSE:
AUC (%5
ALM
0..
10.7
00:10.5
0.0010
0.0 13.0
0.0 10.7
10.0
0.0 10.0
B-ALM
2.220.2
5.0 1009
-L0+100
4.31111
15.7 +2.7
38 10.8
5.0 #5.1
4.0 =5.1
-23.3 23.9
133.00 1:.8
-14212.1
.11.3
12.5
-3.7 10.9
-8.66 1528
11877 52.8
B-QRC
QB-MGP
19.6
10.4 1:0.8
1.077
=1
18.7 42.0
11.7 10.6
12.0 15.7
11.1 =5.7
lowest NLML and RMSE. Overall, these results show that when the GP is inadequate to model the data, both B- B-QBC and QB-MGP perform better than the other acquisition functions.
Multiple inputs To evaluate the performance on higher dimensions, Branin simulator, the strongly non-lincar 3d Ishigami simulator, and the 6d Hartmann simulator with six local minima. BALD underperforms on Branin, but the other acquisition functions have similar performance, with B-QBC having an overall better NLML. For Ishigami, BALD, B-QBC, and QB-MGP reach the best NLML, but the carlier itcrations show that BALD slightly better than the other two acquisition functions. For Hartmann, B-QBC and QB-MGP are the most stable and best terms the NLML, whereas the latter achieves the lowest RMSE.
consider the smooth 2d
5.2 The Overall Performance
From the visual inspection of Figure 4, it is hard to measure how good each the of acquisition functions are. In Table 2, we quantify the performance using the method described earlier, based the relative decrease the under the curve (AUC).

---

First of all. is clear that there not a single acquisition function that is performing the best for all the simulators. B-QBC achieves the largest decrease in AUC for either NIMI. or RMSE in five cases, and QB-MGP is the next best, having the largest decrease four times. This is reflected in the overall performance, where B- B-QBC and QB-MGP are the best performing acquisition functions in terms of the marginal likelihood and rool mean square error, respectively. Oppositely, BALD is the worst performing acquisition function overall, not even better than the baseline, which suggests that this acquisition function might not be suited for Gaussian Processes within the regression setting. Given the overall good consistent performance of B-QBC and QB-MGP (B-QBC only worse than the baseline twice), we say that they robust different complexities in the simulators' outputs.
5.3 Limitations
Gaussian Processes are known to be computationally expensive [Williams and Rasmussen, 2006]. The computational cost scales cubically with the number of data points in the data set, i.e. O(n²), and the GPs are thus only suited for small data sels. A fully Bayesian GP is even more computationally expensive because the MCMC sampling. There exist methods to circumvent this, e.g. variational inference (VI), but often at the cost of the approximation of the joint posterior of the hyperparameters [Lalchand and Rasmussen, 2020]. For future work, we will investigate if the posterior can be approximated by VI, using several initializations. or if it is possible to achieve similar performance by alternating between using fully Bayesian GP and a regular GP.
This paper is based on empirical results that are dependent on specific simulators. The simulators rep- resent diverse and distinct classic homoscedastic simulators and are representative of the engincering problems occurring in the real world [Gramacy, 2020, Sauer et al., 2022, Cole et al., 2022]. However, an interesting case that is less often investigated in the literature on active learning with simulators, is simulator with heteroscedastic noise.
A common heteroscedastic case study is the motorcycle data set [Silverman, 1985, Gramacy and Lee, 2008, Gramacy, 2020J. We create a corresponding simulator by fitting a variational GP [Hensman et al., 2015] to the motorcycle accident data. For reproducibility, the mean and standard deviation of the simulator is given in appendix (A.7). The experiments the Motorcycle simulator explore how the active learning acquisition functions perform when the simulator has heteroscedastic noise, but we model it with a homoscedastic GP. The simulator and the results are seen in Figure 5. Most conspicuous is the poor and good performance B-QBC and QB-MGP, respectively. In appendix A.8, we show that B-QBC is misled by the heteroscedastic noise and focuses too much on the disagreement in the middle, and that the B-ALM component of QB-MGP acts as diversity measure that encourages more exploration since it aims reducing the overall predictive uncertainty.
Motoreycle
Motorcycle
1.00
ALM B-ALM
NLME
80.75
BALD
0.60
B-OBC QB 8-MGP
0.25
10
20
30
40
10
20
30
Figure 5: Left: Visualization of the Motorcyele simulator. Right: Performance across 10 runs =1 standard deviation. The æ-axis represent the number of itcrations.
Another aspect of the work in this paper is the use of the domain and expert knowledge. The incorporation of the domain and expert guidance regarding the simulator under study can be decisive factor a successful active learning strategy. However, many practical situations, such priori domain expertise may not be readily accessible or even translatable into the functional structure of the model useful modeling information. On these occasions, generic tools that are robust enough handle a plethora of diverse simulation output behaviors are prudently advisable. If we had such information regarding the functional complexity, e.g., knowing that the signal is periodic, this study does not show which of the acquisition functions is best. In future work, it would indeed be interesting to see if B-QBC and QB-MGP would perform equally well.

---

6 Conclusion
In this paper, we propose two active learning acquisition functions: Bayesian Query-by-Cone Coymmiitee (B-QBC) and Query by Mixture of Gaussian Processes (QB-MGP), both of which are suited for fully Bayesian GPs. They designed explicitly handle the well-known bias-variance trade-off by optimization of the GP's two hyperparameters. length scale and noise-term. We empirically show that they query new data points more efficiently than previously used acquisition functions. Across six classic simulators, which cover different complexities and numbers of inputs, show that B-QBC and QB-MGP are the two functions that achieve the best marginal likelihood and root mean square error, respectively, with the fewest iterations. On average, across the simulators, B-QCB reduced the negative marginal log-likelihood with 41 41%, and QB-MGP decreased the root mean square error with 12% compared to the baseline. To this end, we believe that the proposed acquisition functions are robust enough to handle a variety of diverse simulation output behaviors, while being entirely independent of any prior understanding of the underlying output distributions of the simulator.
Acknowledgments and Disclosure of Funding
This work was supported by NOSTROMO, framed in the scope of the SESAR 2020 Exploratory Research lopic SESAR-ER ER4-26-2019, funded by SESAR Joint Undertakingthrough the European Union's 's Horizon 2020 research and innovation programme under grant agreement No 892517.
References
F. Antunes, B. Ribeiro. F. C. Pereira, and R. Gomes. Efficient Transport Simulation With Restricted Batch-Mode Active Learning. IEEE Transactions on Intelligent Transportation Systems, 19(11):36423361, nov 2018. ISSN 1524-9050. doi: 10.1 10.1109/TTTS.211 28.2224555. URL htwph:tttp//tttt pttttp:///eeeexporee.eeee htt.://ppprr...ee..rroo/ooeeeeee org/document/ 8419064/.
E. Bingham, J. Chen, M. Jankowiak, F. Obermeyer, Pradhun, T. Karaletsos, R. Singh, P. Szerlip. P. Horsfall, and N. Goodman. Pyro: Deep universal probabilistic programming. The Journal of Machine Learning Research, 20(1):973 978, 2019. URL https: //jmrr...///////// csail. mit...mmt.......mmmmmmamm.mma..e www.pppers// 403.
C. M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006. ISBN 978-0387-31073--2 URL http://research. mi cicosoft..o cwm/ coam/en-uss en-us/um /as//m//ppt wwwpppppp////eeeet/eoooooeeeeeee e/ cmbishop/prml/
R. Burbidge, J. Rowland, and R. D. King. Active learning for regression based on query by committee. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelfigence and Lecture Notes in Bioinformatics). 4881 LNCS:209-218, 2007. ISSN 161 13349. dui: 10.10077/78---540-77000
22.
W. Cai, Y. Zhang, and Zhou. Maximizing expected model change for active learning in regression. Proceedings IEEE International Conference on Data Mining, ICDM, pages 51-60, 2013. ISSN 15504786. doi: 10. 00.10091CDM.2013..000
S. Chabanet, H. El-Haouzi, and P. Thomas. Coupling digital simulation and machine learning metamodel through an active learning approach in industry 4.0 context. Computers in Industry, 133:103529, 2021.
T. Chen and J. Ren. Bagging for Gaussian process regression. Neurocompuring, 1007-9):100--1110, 2009. ISSN 09252312. doi: 10.1016/j 6...0061...uumm2200000 2008.00..0000000000 2008.090000222220000000000000000
D. A. Cole, R. B. Gramacy, J. E. Warner, G. F. Bomarito, E. Leser, and W. P. Leser. Entropy-based aduptive design for contour finding and estimating reliability. Journal of Quality Technology, pages 1-18, 2022. URL https:///ol
org/ 10. 1080/00224065. 2022. 2053795.
F. L. Fernandez, L. Martino, V. Elvira, D. Delgado, and J. Lopez-Santiago. Adaptive Quadrature Schemes for Bayesian Inference via Active Learning. IEEE Access, 8:208462 208433, 2020. ISSN 21693536, doi: 10. 10.1109/ACESSS SRDEEESS.202...00 3020.3038333.
J. R. Gardner, G. Pleiss, D. Bindel, K. Q. Weinberger, and A. G. Wilson. Gpytorch: Blackbox matrix- matrix-matrix Gaussian process inference with GPU acceleration. Advances in Neural Information Processing Systems, 31; 7586-7586, 2018 ISSN 10495258.
Gorissen, Crombecq, I. Couckuyt, and Dhaene, Automatic approximation of expensive functions with active learning. In Foundations of Computational, Intelligence Volume 1, puges 35-62. Springer. 2009.

---

R. B. Gramacy, Surrogates. Chapman and Hall/CRC, mar 2020. ISBN 9780367815493. doi: 10.1201/
9780367815493. URL htt.ttttp///eee http:////wwwttlofffaaaee taylorfrancis com/ bow.s////////000///////000
R. B. Gramacy and H. K. Lee. Bayesian treed gaussian process models with an application to computer modeling, Journal of the American Statistical Association, 103, 2008. ISSN 01621459. doi: 10.1 016214508000000689.
R. B. Gramacy and H. K. Lee. Adaptive design and analysis of supercompuler experiments. Technometrics. 51 (2):130-145, 2009. ISSN 00401706, doi: 10.1 19..T9HHHH.200 201500000.... URL httpp:://oi....ooo org/1........... 1198/ TECH.2009 2009.2009.0000 0015.
R. B. Gramacy and H. K. Lee. Cases for the nugget in modeling computer experiments. Statistics and Computing, 22, 2012. ISSN 1960 doi: 10.1007/s 0222-010-9224--
C. Guestrin, A. Krause, and A. P. Singh. Neur-optimal sensor placements in Gaussian processes. In Proceedings of the 22nd international conference Machine learning ICML '05, volume 1, pages 265-272, New York, New York, USA, 2005. ACM Press. ISBN 1595931805. doi: 10.1145/1 102351. URL
http://portal.ce acm. org/ www.t/iiiaiiioon//ooo dtfmmmdood-11022555 1102385.
Hensman, A. Matthews, and Z. Ghahramani. Scalable variational gaussian process classification. Artificial Intelligence and Statistics, pages 351-360. PMLR. 2015.
M. D. Hoffman and A. Gelman. The no-u-turn sampler: adaptively setting path lengths in hamiltonian monte carlo. J. Mach. Learn Res., 15(1):1593-162 2014.
N. Houlsby, F. Huszár, Z. Ghahramani, and M. Lengyel. Bayesian Active Learning for Classification and Preference I Learning. arXiv, pages 1-17, 2011. URI. http://arxii wwwg////////////eoo wttps//112....11111 .5755.
N. Houlsby, Huszar, Z. Ghahramani, and Hernández lobato. Collaborative gaussian processes for preference learning. Advances in Neural Information Processing Systems, 25, 2012.
B. H. Iswanto. Active learning by increasing model likelihood for Gaussian mixture models based classifiers. 1OP Conference Series: Materials Science and Engineering, 1098(3)003036, mar 2021. ISSN 1757- 8981. doi: 10. 1088010877778999 (088/1757-899/ 109009990 0//////3300 3/032036.
URL https://Lopsciencce iipeeeii.p.o.... cup.oogg/rticce/10
A. Keane, A. Forrester, and A. Sobester. Engineering Design via Surrogate Modelling: A Practical Guide John Wiley & Sons, 2008. doi: 10.2514/4.479557.
A. Kirsch, van Amersfoort, and Y. Gal. BatchBALD: Efficient and diverse butch acquisition for deep Bayesian active learning, Advances in Neural Information Processing Systems, 32, 2019. ISSN 10495258.
A. Krogh and J. Vedelshy. Neural network ensembles, cross validation, and active learning. In Advances in Neural Information Processing Systems, volume 7, pages 231-2334 1995.
V. Lalchand and C. E. Rasmussen. Approximate inference for fully bayesian gaussian process regression. In Symposium Advances Approximate Bayesian Inference, pages 1-12. PMLR, 2020.
D. C. MacKay. Information- Basse Objective Functions for Active Data Selection. Neural Computation, 4(4): 590-604, 1992. ISSN 0899-7667. doi: 10.1 (122///co.199..444....
A. Marrel, B. looss, B. Laurent, and O. Roustant. Calculations of sobol indices for the gaussian process metamodel. Reliability Engineering and System Safety, 94, 2009. ISSN 0951 19318320, doi: r0.0 07.008.
J. O'Neill, Jane Delany, and B. MacNamee. Model-Free and Model-Based Active Learning for Re- gression. In Advances computational intelligncee systems, pages 375-386. Springer, 2017. ISBN 9783319465623. doi: 00.10077998-33319-46666 24. URL http://tt www.iii...rinnn....eee cot.//..c.mm0.0./0...1.1.......0 978-3-311-44562-33
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-leamn: Machine learning in Python. Journal of Machine Learning Research. 12:2825 -8330. 2011.
V. Picheny, T. Wagner, and D. Ginshourger. A benchmark of triging-based infill criteria for noisy optimization. Structural and Mutidisciplinary Optimization, 48, 2013. ISSN 16151 488. doi: 10. 10.10007/0015--00 558-013-0919-1
T. RayChaudhuri and I.. G. Hamey. Minimization data collection by active learning. IEFF International Conference on Neural Networks Conference Proceedings. 3:1338-1341, 1995. doi: 10.1 109//nn...n... 1995.. 487351.

---

C. Ris,, Antunes, Gurtner, F. C. P.reeiia, L. Delgado, and C. M. Lima Azevedo. Active learning metamodels for atm simulation modeling. In Proceedings of the 11th SESAR Innovation Days, 2021, 2021. URI. https: //ww.//wwaaaaao//. sesaa ju. eu/SIDs2021.
A. Sauer, R. B. Gramacy, and D. Higdon, Active learning for deep gaussian process surrogates. Technometrics, 000):1-15, 2022. doi: 10. 10110000000000.0200 2008550. URL https://doi. org/10 /0/ 1080/00401706. 2021 2008505.
B. Settles. Active learning literatrre survey. University of Wisconsin-Madison Department of Computer Sciences, 2009.
H. S. Seung. M. Opper. and H. Sompolinsky. Query committee. In Proceedings of the fifth annual workshop Computational learning theory COLT 92. pages 287-294, New York. New York, USA, 1992. ACM Press. ISBN 0897914977 497X. doi: 1..1145/130388.134411 URL http://portal. org/ citation.
cfm?doid=130388. 130417.
B. W. Silverman. Some aspects of the spline smoothing upproach to non-parametric regression curve fitting. Journal of the Royal Statistical Society: Series B (Methodological), 47, 1985. ISSN 0035-9246. doi:
10.1111/2511-6661.199 091777666611119...11. 1988..b01127...
Snoek, H. Larochelle, and R. P. Adams. Practical bayesian optimization machine learning algorithms. Advances in Nearal Information Processing Systems, volume 2012.
S. Surjanovic and D. Bingham. Virtual library of simulation experiments: Test functions and datasets. Retrieved June 3, 2022, from http://www sfu. ca/"ssur 2022.
Svendsen, Martino, and G. Camps-Valls Active emulation of computer codes with Gaussian processes Application remote sensing. Puttern Recognition, 100000, 2020. ISSN 00313203. doi: 10.10 patcog.2019. 1000119.1111000 107103. URL https: //doi
1016 patcog. 2019. 107103.
G. Van Kempen and L. Van Vliet. Mean and variance of ratio estimators used in fluorescence ratio imaging. Cytometry: The Journal of the International Saciety for Analytical Cytology, 3944):00--305 2000.
C. Williams Rasmussen. Gaussian processes regression. Advances Neural Information Processing Systems, 8, 1995.
C. K. Williams and C. E. Rasmussen. Gaussian processes for machine learning, volume 2. MIT press Cambridge. MA, 2006.
press
D. Wu. Pool-Based Sequential Active Learning for Regression. Transactions on Neural Networks and Learning Systems, 30(5): 348-1359, 2018, ISSN 23318422.
D. Wu, C. T. Lin, and Huang. Active learning for regression using greedy sampling. Information Sciences, 74090-105, 2019. ISSN 00200255. doi: 10.1066/j.ns.200 201.ii.......0000000 URI. http:::/pt.p..orrg//eeee. 1016/ ins. 2 2018.0088000 0900000.
Y. Yang and M. Long. A benchmark and comparison of active learning for logistic regression. Patter Recognition. 83:401-415, 2018.
Y. Yao, A. Vehtari, and A. Gelman. Stacking for non-mixing bayesian computations: The and blessing of multimodal posteriors. Journal of Machine Learning Research, 23(79):1-45, 2022. URL http: attp::/jmmr. hts//apprrs/v333
1426.
J. Zhao, S. Sun, H. Wang. and Z. Cao. Promoting active learning with mixtures of Gaussian processes. Knowledge-Bass Baxed Systems, 188: 105044, 2020. ISSN 09507051. doi: 10. 1o.1166..ossss.2119 105044. URL https://doi org/ httg//01111.. 1016/j. knosys. 2019. 105044.

---

Checklist
For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? [Yes] All the claims in the abstract and introduction is reflected in the discussion of the results, cf. section 5.
(b) Did you describe the limitations of your work? [Yes] See subsection 5.3
(c) Did you discuss any potential negative societal impacts of your work? [N/A] No, we create a generic active learning acquisition function, so there is no direct negative societal impacts.
(d) Have you read the ethics review guidelines and ensured that your paper conforms them? [Yes] The work presents generic algorithm that is tested on simulators and does not have any ethic related issucs.
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A] We give no thcoretical results.
(b) Did you include complete proofs of all theoretical results? [N/A] We give theoretical results.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed to reproduce the main experi- mental results (either in the supplemental material or as a URLY? [Yes] See section 5.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes] See paragraph "Experimental settings" in section 5.
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes] See both error bars in Figure and Table
(d) Did you include the total amount compute and the type of resources used (e.g.. type of GPUs, internal cluster, or cloud provider)? [Yes] See paragraph "Experimental settings" in section 5.
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes] We cite GPyTorch Torch [Gardner al., 2018] and Pyro [Bingham et al., 2019] as well as the used simulators. See Table 1.
(b) Did you mention the license of the asscts? [No] All of is open-source software.
(c) Did you include any new assels either in the supplemental material or as URL? [No] We do not provide any
assets, only the code for reproducing the experiments. (d) Did you discuss whether and how consent was obtained from people whose data you't 're using/curating? [No] All the simulators are previously used in the literature and are available at https: https://wwzrr /www. .fu. ca/ jssrjjno//
(e) Did you discuss whether the data you are using/curating contains personally identifiable information or offensive content? [N/A]
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given participants and screenshots, if applicable? [N/A]
(b) Did you describe any potential participant risks, with links Institutional Review Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent on participant compensation? [N/A]


Reviews:
[{"review_id": "w-DhAIPTe5n", "reviewer": "Reviewer_rWt1", "summary": "This paper focuses on active learning using Gaussian Processes, where the authors take a fully Bayesian approach to optimize the bias-variance trade-off, an important problem in various learning problems.\nEspecially, this work focuses on a small-data setting, where such bias-variance trade-off can significantly affect the learning outcomes.\nThe authors propose two acquisition functions based on FBGP for regression tasks, showing the potential advantages of the proposed schemes.\n", "questions": "1. Please elaborate on the major advantages of the proposed active learning schemes in a small-data setting, especially, based on the results shown in Figure 4, since it is unclear based on the current evaluation results.\n\n2. Please compare the computational cost of the proposed methods, in comparison with other schemes considered in this study.\n\n3. Please clarify whether and how the proposed schemes can be generalized beyond the tasks/settings considered in the current study.\n\n4. How would the proposed schemes (especially, QB-MGP) compare to other active learning schemes (such as ELR) that focus on predictive performance?\n\n5. Please clarify what would be the main potential advantages of the proposed schemes compared to other recent developments in Bayesian active learning.\n\n \n\n\n", "limitations": "The authors note that they \"create a generic active learning acquisition function\" and therefore \"there is no direct negative societal impacts.\"\n", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Taking a Bayesian approach to automatically balance the bias-variance trade-off and improve the robustness of the learning outcomes and the predictive performance of the learned models in a small-data setting is expected to be beneficial, and this work demonstrates the potential advantages of using FBGP  for active learning in regression tasks.\n\n1. Table 2 and Figure 4 show that the two proposed acquisition functions - B-QBC and QB-MGP - can lead to better regression results for a number of test cases. However, these results also show that the proposed scheme do not consistently outperform other alternatives, nor significantly improve the learning outcomes when they do. Especially, while the comparison based on AUC (in Table 2) make the performance improvement attained by the proposed schemes relatively prominent in some cases, the actual improvement shown in the curves (Figure 4) do not appear to be very significant. This is especially so when the data size is relatively small, even though that is the setting that motivates the current work.\n\n2. The computational burden of taking a fully Bayesian approach is not discussed in detail, and the authors simply mention that they assume that the computational burden for querying new data points would far exceed that for the Bayesian inference. While this may be the case in some applications, it would be nevertheless important and informative to compare the computational cost of different active learning schemes, which is absent in the current study.\n\n3. The authors do not discuss the general applicability of the proposed scheme beyond the regression tasks considered in the current work. For example, BALD - shown to outperform the proposed scheme in a number of cases - is widely used for active learning for classification, and it would be meaningful to present how the proposed schemes would be applied to classification tasks and how their performance would compare to BALD and other alternatives.\n\n4. While the authors proposed QB-MGP to consider the \"predictive performance\" of the learned model, when the prediction performance is of main interest, it would be more sensible to compare the performance with other active learning schemes that focus on this aspect. For example, ELR (expected loss reduction) strategies are widely used to acquire new data points that are expected to optimally reduce the error rather than reducing the uncertainty of the model parameters.\n\n5. Recently, a number of Bayesian active learning schemes have been proposed based on the ELR strategy, whose performance has been shown to outperform BALD and various other methods with theoretical convergence guarantees. Some recent examples include:\n\n(1) Tan et al, Diversity Enhanced Active Learning with Strictly Proper Scoring Rules, NeurIPS 2021.\n(2) Zhao et al, Efficient Active Learning for Gaussian Process Classification by Error Reduction, NeurIPS 2021.\n\nConsidering that the aforementioned Bayesian active learning methods have shown to consistently outperform BALD and other existing methods with the added benefit of convergence guarantee to the optimal model, the potential benefits of the proposed schemes remain somewhat unclear. Further comparison and elaboration would be necessary to clarify the most significant advantages of the proposed methods, in comparison with the latest relevant developments in Bayesian active learning.\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "jIBkjt8JyTe", "reviewer": "Reviewer_8dtu", "summary": "This work proposes the use of fully Bayesian GPs in the context of Bayes opt and active learning. They propose fully Bayesian versions of known acquisition functions that leverage information from the hyperparameter posterior and also suggest using the posterior predictive variance of the  GMM (non-Gaussian) process to derive an acquisition function. The claim is that the Bayesian variants yield more effective acquisition functions --  implicitly because they approximate the posterior predictive uncertainty better than ML-II. They demonstrate and analyse the performance across methods on 6 simulator tasks", "questions": "1) Isn't the most reasonable mode given by the mode where the marginal likelihood is maximized? and typically, the modes manifest in very starkly different marginal likelihood values. \n2) The underperformance of B-QBC on the heteroscedastic motorcycle data is not sufficiently explained, QB-MGP performs best but in terms of the criteria is not so different than B-QBC.\n3) Both B-QBC and QB-MGP use hyper samples from the posterior - then \\mu_GMM is the same as the mean of the models corresponding to thetas drawn from the hyper posterior \\bar{\\mu}? Their performance should be very similar but they are quite different - is there more insight on this? ", "limitations": "The authors do discuss limitations - but conspicuously miss out the obvious one which is the compute for FBGPs. ", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n* Interesting interpretation of the multi-modal marginal likelihood surface in terms of the bias-variance tradeoff parlance.  \n* Good presentation, well-written and clear. \n* Decent experimental evaluation.\n* Approximating the hyperparameter posterior - a relatively understudied area in literature.\n\nWeakness:\n\n*A major concern is the computational cost of running FBGP at each iteration. A naive strategy (albeit sufficient for small datasets like the ones studied) would scale as O(MN^{3}) where M is the number of samples - how exactly can this method scale? perhaps by interleaving FBGP at intervals with traditional ML-II optimisation - a more sophisticated strategy would be required. \n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "-GsYk7QWZc6", "reviewer": "Reviewer_u8Av", "summary": "The authors propose a novel active learning scheme for GPs based on a full-Bayesian solution.", "questions": "I have only a suggestion: to complete the  state-of-the-art discussion,\n including the acquisition functions in GP schemes (for regression of for quadrature) that considers also the gradient information as suggested in\n\nD. H. Svendsen, et al. Active Emulation of Computer Codes with Gaussian Processes - Application to Remote Sensing, Pattern Recognition Volume 100, 2020,\n\nF. Llorente, et al, \"Adaptive quadrature schemes for Bayesian inference via active learning\", IEEE Access, Volume 8, 2020. \n\nM. Kanagawa and P. Hennig, “Convergence Guarantees for Adaptive Bayesian Quadrature Methods,” in Advances in Neural Information Processing Systems, 2019, pp. 6234–6245.", "limitations": " The contribution seems a bit incremental.", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "The paper is well-written and it is nice to read this work. This is the the main strength, in my opinion.\nThe state-of-the-art discussion is the nicest part, in my opinion.\n\nWeakness: the contribution seems a bit incremental.", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}]

Author rebuttal:

Decision: Accept