Bayesian Active Learning with Fully Bayesian Gaussian Processes

1 INTRODUCTION . Gaussian Processes ( GPs ) are the canonical models to use for Bayesian optimization and metamodeling ( Snoek et al. , 2012 ; Gramacy , 2020 ) . GPs are well-known for their ability to deal with small to medium size data sets as well as balancing complexity and regularization - together with their inherent ability to handling uncertainties - this makes them ideal in such applications ( Williams & Rasmussen , 2006 ) . In both cases , often only limited data is accessible so the natural balance between complexity and regularization helps prevent severe overfitting , additionally making the model flexible enough to model nonlinear functions . Likewise , the quantification of uncertainty is commonly used in the acquisition function to guide the Bayesian optimization algorithms and the active learning schemes that are almost inevitable to efficiently build a metamodel . On the other hand , it is not a flawless procedure to use GPs to guide the two schemes . The same GP is used as both predictor and guide , and thus the choice of predictor will affect the guide , and vice versa . Trivially , the more data there is available for modeling , the less pronounced this problem will be . However , in the context of both Bayesian optimization and active learning , where the data sets tend to be rather small , a wrong predictor can result in misguidance , thus hindering the performance and efficiency . In this paper , we mitigate this problem by not only focusing on a single predictive model but consider many predictive models through multiple model hypotheses at once . A GP is typically fitted through evaluation of the marginal likelihood , which automatically incorporates a trade-off between complexity and regularization ( Williams & Rasmussen , 2006 ) . However , when the data is scarce , it is more challenging to choose the appropriate trade-off , and different configurations of the hyperparameters of the GP can give rise to distinct fits . This is illustrated in Figure 1 , where two seemingly reasonable fits might guide the schemes quite differently . The problem is directly related to the well-known bias-variance trade-off , although we reformulate it as a balance between modeling the data as signal or noise . For the common stationary covariance functions of GPs , e.g . the radial-basis function or the Matérn class of functions , together with a Gaussian High noise regime Low noise regime Figure 1 : Two GPs with different hyperparameters . The left plot shows a GP with high noise and a long length scale , and the right plot shows a GP with low noise and a short length scale . likelihood , this is directly reflected in the two hyperparameters length scale ` and noise σ2ε . If the data is modeled primarily as noise , both σ2ε and ` are large , whereas if the data is modeled primarily as a signal , both σ2ε and ` are small . In the case of limited data , the joint posterior distribution of the two hyperparameters is likely to be characterized by two modes , as illustrated in Figure 2 for the data in Figure 1 . In that case , it is difficult to tell whether it is best to model the data as noise or signal , and a wrong choice of mode will imply non-optimal guidance from the acquisition function . The literature suggests handling this problem with clever initializations of the hyperparameters Williams & Rasmussen ( 1996 ) or by favoring small ` and σ2ε by either always initializing hyperparameters in the low noise regime or by applying strong priors ( Gramacy , 2020 ) . However , none of these approaches directly address the core problem : which mode to choose ? Ideally , this should be answered with prior information about the problem , although typically that is not available , making these approaches less practical . In this paper , we follow a general approach and assume no prior knowledge about the kernel or hyperparameters . We consider multiple model hypotheses by replacing the fitting procedure of the marginal likelihood with Markov Chain Monte Carlo ( MCMC ) sampling . Lalchand & Rasmussen ( 2020 ) show that with fixed , medium-sized data sets and carefully chosen kernels , it is beneficial to fit GPs with MCMC instead of by maximizing the marginal likelihood . We show that the same is true for very small data sets and with a general kernel . Our main contribution is the proposal of two new acquisition functions for active learning that utilizes the extra information from the hyperparamters ’ posteriors estimated by MCMC to seek the most reasonable mode alongside minimizing the predictive variance . We show that the two acquisition functions are more accurate and robust than other common functions across multiple benchmark simulators used in the related literature . 2 RELATED WORK . The two proposed acquisition functions are specifically designed for active learning schemes for regression tasks . In this section , we lay out the related work , although the specific acquisition functions used for comparison purposes in the experiments are described in detail in section 4 . Further , in this section , we cover the essentials of Query-by-Committee and Gaussian Mixture Models , as these constitute the backbone of the proposed acquisition functions . Active Learning The main idea of active learning ( AL ) is to actively choose a new data point to label and add to the current training data set ( Settles , 2009 ) . In the context of metamodeling , new data is often added sequentially , i.e. , one data point at a time ( Gramacy , 2020 ) , but in other applications , it can be beneficial to query batches of data instead ( Kirsch et al. , 2019 ) . The acquisition functions can be divided into model-based and model-free functions , where the former utilize information from the model and the latter do not ( O ’ Neill et al. , 2017 ) . Both types of functions seek to minimize the expected predictive loss of the model . Another approach is to minimize the number of possible models . Houlsby et al . ( 2011 ) divide the active learning acquisition functions into being based on either decision or information theory . Decision-based functions seek to minimize the expected predictive loss of the model in the hope of maximizing the performance on the test set . Information-theoretic-based functions instead try to reduce the number of possible models , for example , through the KL-divergence or Shannon entropy ( Houlsby et al. , 2011 ) . It is not straightforward to use information-theoretic acquisition functions . However , if you have access to the posterior of the model ’ s parameters , Houlsby et al . ( 2011 ) have derived the algorithm Bayesian Active Learning by Disagreement ( BALD ) , which can be applied in general . Generally , BALD seeks the data point that maximizes the decrease in the expected posterior entropy of the parameters . Query-by-Committee The Query-by-Committee ( QBC ) is a specific acquisition function that was originally proposed for classification tasks ( Seung et al. , 1992 ) . It aims to maximize the disagreement among the committee to get the highest information gain and minimize the version space , which is the set of model hypotheses aligned with the training data . The construction of the committee is the core component of QBC since it is the committee ’ s ability to accurately and diversely represent the version space that gives rise to informative disagreement criteria ( Settles , 2009 ) . QBC can also be applied for regression problems . Krogh & Vedelsby ( 1995 ) construct the members of the committee by random initializations of the weights in the neural networks . RayChaudhuri & Hamey ( 1995 ) apply bagging and train the members on different subsets of the data set . In general , QBC constructed by bagging has been used as a benchmark with mixed results ( Cai et al. , 2013 ; Wu , 2018 ; Wu et al. , 2019 ) . Burbidge et al . ( 2007 ) show that the less noise there is in the output , the better QBC is compared to random querying . They also highlight the fact that with a misspecified model , QBC might perform worse than random querying . None of these approaches explore the usage of MCMC samples of the posterior to construct a committee . To the best of our knowledge , we are the first to propose QBC based on model hypotheses drawn from the posterior . Gaussian Process as a Gaussian Mixture Model Mixture models have recently been applied in active learning for classification tasks . Iswanto ( 2021 ) propose to use Gaussian Mixture Models ( GMMs ) with active learning , where they design a specific acquisition function that queries the data point that maximizes the expected likelihood of the model . Zhao et al . ( 2020 ) use a mixture of GPs in active learning , where each component is fitted to a subset of the training set . The combination of GMMs and GPs have previously been explored for static data sets . Chen & Ren ( 2009 ) investigate regression tasks and apply bagging , where they repeatedly randomly sample data points from the training set to construct new subsets to get GPs fitted to different data . Among other combination rules , they combine the predictive posteriors of these GPs into GMMs . They obtain even better performance by weighting the models by the predictive uncertainty such that models with high uncertainty are given smaller weights , and vice versa . Instead of using bagging , we construct the multiple GPs by using the MCMC samples of the hyperparameters ’ joint posterior , and then obtain a natural weighting of the GPs : the GMM will consist of more GPs with hyperparameters close to the modes than hyperparameters far away . The formal procedure is given in section 4 . To the best of our knowledge , we are the first to combine GMMs and GPs in this manner . 3 GAUSSIAN PROCESSES . The Gaussian Processes ( GPs ) are the central models in this work . In this section , we give a brief overview of GPs before covering the Fully Bayesian GPs . For a thorough description of GPs , we refer to Williams & Rasmussen ( 2006 ) . Gaussian Processes A Gaussian Process ( GP ) is a stochastic function fully defined by a mean function m ( · ) and a covariance function ( often called a kernel ) k ( · , · ) . Given the data ( X , y ) = { xi , yi } Ni=1 , where yi is the corrupted observations of some latent function values f with Gaussian noise ε , i.e. , yi = fi + εi , εi ∈ N ( 0 , σ2ε ) , a GP is typically denoted as GP ( mf ( x ) , kf ( x , x′ ) ) . It is common practice to set the mean function equal to the zero-value vector and thus , the GP is fully determined by the kernel kf ( x , x′ ) . For short , we will denote the kernel Kθ , which explicitly states that the kernel is parameterized with some hyperparameters θ . Given the optimal hyperparameters , the predictive posterior for unknown test inputs x∗ is given by p ( f∗|θ̂ , y , X , X∗ ) = N ( µ∗ , Σ∗ ) with µ∗ = K∗θ ( Kθ + σ 2 εI ) −1 y and Σ∗ = K ? ? θ −K∗θ ( Kθ + σ 2 εI ) −1 K ? > θ ( 1 ) where K ? ? θ denotes the covariance matrix between the test inputs , and K ∗ θ denotes the covariance matrix between the test inputs and training inputs . Covariance matrix We use the canonical kernel automatic relevance determination ( ARD ) Radial-basis function ( RBF ) given by k ( x , x′ ) = exp ( −||x− x′||2/2 ` 2 ) where ` is a vector of length scales ` 1 , ... , ` d , one for each input dimension . Often the kernel is scaled by an output variance but here we fix it to one and solely focus on the two other hyperparameters : length scale and noise-term . The noise-term σ2ε is integrated into the kernel with an indicator variable by adding the term σ2εI { x=x′ } to the current kernel . Fully Bayesian Gaussian Processes ( FBGP ) A FBGP extends a GP by putting a prior over the hyperparameters θ ∼ p ( θ ) and approximate their full posteriors . The joint posterior is then given by p ( f , θ|y , X ) ∝ p ( y|f ) p ( f |θ , X ) p ( θ ) ( 2 ) and the predictive posterior for the test inputs X∗ is p ( f∗|y ) = ∫∫ p ( f∗|f , θ ) p ( f |θ , y ) p ( θ|y ) dfdθ ( 3 ) where the conditioning on X and X∗ have been omitted for brevity . The inner integral reduces to the predictive posterior given by a normal GP . However , the outer integral remains intractable and is approximated with MCMC inference as p ( f∗|y ) = ∫ p ( f∗|y , θ ) p ( θ|y ) dθ ' 1 M M∑ j=1 p ( f∗|y , θj ) , θj ∼ p ( θ|y ) ( 4 ) As well known in machine learning modeling , there is no free lunch . Adapting the hyperparameters of a FBGP is computationally expensive compared to the approach with GPs and maximum likelihood estimates . However in Bayesian optimization and active learning , the computational burden for querying a new data point will often be of magnitudes higher . Evaluation and Fitting of GPs A GP is typically fitted by maximizing the marginal likelihood with gradient descent , where the marginal likelihood can be computed analytically . For the FBGP , the inner integral is intractable and is therefore optimized with approximate inference using the selftuning method of Monte Carlo Markov Chain ( MCMC ) called the No-U-Turn-Sampler ( NUTS ) ( Hoffman & Gelman , 2014 ) . The MCMC inference makes the full posterior of the hyperparameters available , making it possible to extend the point estimates to a joint distribution . In the experiment , we apply acquisition functions that use the full posterior distribution ; however , since the experiments are designed to benchmark the acquisition functions , we always use the mode of the joint posterior for prediction and evaluation .

Summary: This paper takes a fully Bayesian approach to Gaussian Process (GP) active learning by using MCMC sampling to consider multiple model hypotheses from a full posterior, from which it selects examples for GP regression using two new active learning strategies --- Bayesian Query-by-Committee (B-QBC) and Query by Mixture of Gaussian Processes (QB-MGP). B-QBC queries the data point that maximizes the disagreement between the sampled models’ mean values. The idea is that this should provide the highest information about the optimal posterior mode. QB-MGP uses a combination of B-QBC with entropy sampling. The authors evaluate these strategies against several baseline active learning methods on several simulators.