Bayesian Active Learning with Fully Bayesian Gaussian Processes

1 INTRODUCTION . Gaussian Processes ( GPs ) are the canonical models to use for Bayesian optimization and metamodeling ( Snoek et al. , 2012 ; Gramacy , 2020 ) . GPs are well-known for their ability to deal with small to medium size data sets as well as balancing complexity and regularization - together with their inherent ability to handling uncertainties - this makes them ideal in such applications ( Williams & Rasmussen , 2006 ) . In both cases , often only limited data is accessible so the natural balance between complexity and regularization helps prevent severe overfitting , additionally making the model flexible enough to model nonlinear functions . Likewise , the quantification of uncertainty is commonly used in the acquisition function to guide the Bayesian optimization algorithms and the active learning schemes that are almost inevitable to efficiently build a metamodel . On the other hand , it is not a flawless procedure to use GPs to guide the two schemes . The same GP is used as both predictor and guide , and thus the choice of predictor will affect the guide , and vice versa . Trivially , the more data there is available for modeling , the less pronounced this problem will be . However , in the context of both Bayesian optimization and active learning , where the data sets tend to be rather small , a wrong predictor can result in misguidance , thus hindering the performance and efficiency . In this paper , we mitigate this problem by not only focusing on a single predictive model but consider many predictive models through multiple model hypotheses at once . A GP is typically fitted through evaluation of the marginal likelihood , which automatically incorporates a trade-off between complexity and regularization ( Williams & Rasmussen , 2006 ) . However , when the data is scarce , it is more challenging to choose the appropriate trade-off , and different configurations of the hyperparameters of the GP can give rise to distinct fits . This is illustrated in Figure 1 , where two seemingly reasonable fits might guide the schemes quite differently . The problem is directly related to the well-known bias-variance trade-off , although we reformulate it as a balance between modeling the data as signal or noise . For the common stationary covariance functions of GPs , e.g . the radial-basis function or the Matérn class of functions , together with a Gaussian High noise regime Low noise regime Figure 1 : Two GPs with different hyperparameters . The left plot shows a GP with high noise and a long length scale , and the right plot shows a GP with low noise and a short length scale . likelihood , this is directly reflected in the two hyperparameters length scale ` and noise σ2ε . If the data is modeled primarily as noise , both σ2ε and ` are large , whereas if the data is modeled primarily as a signal , both σ2ε and ` are small . In the case of limited data , the joint posterior distribution of the two hyperparameters is likely to be characterized by two modes , as illustrated in Figure 2 for the data in Figure 1 . In that case , it is difficult to tell whether it is best to model the data as noise or signal , and a wrong choice of mode will imply non-optimal guidance from the acquisition function . The literature suggests handling this problem with clever initializations of the hyperparameters Williams & Rasmussen ( 1996 ) or by favoring small ` and σ2ε by either always initializing hyperparameters in the low noise regime or by applying strong priors ( Gramacy , 2020 ) . However , none of these approaches directly address the core problem : which mode to choose ? Ideally , this should be answered with prior information about the problem , although typically that is not available , making these approaches less practical . In this paper , we follow a general approach and assume no prior knowledge about the kernel or hyperparameters . We consider multiple model hypotheses by replacing the fitting procedure of the marginal likelihood with Markov Chain Monte Carlo ( MCMC ) sampling . Lalchand & Rasmussen ( 2020 ) show that with fixed , medium-sized data sets and carefully chosen kernels , it is beneficial to fit GPs with MCMC instead of by maximizing the marginal likelihood . We show that the same is true for very small data sets and with a general kernel . Our main contribution is the proposal of two new acquisition functions for active learning that utilizes the extra information from the hyperparamters ’ posteriors estimated by MCMC to seek the most reasonable mode alongside minimizing the predictive variance . We show that the two acquisition functions are more accurate and robust than other common functions across multiple benchmark simulators used in the related literature . 2 RELATED WORK . The two proposed acquisition functions are specifically designed for active learning schemes for regression tasks . In this section , we lay out the related work , although the specific acquisition functions used for comparison purposes in the experiments are described in detail in section 4 . Further , in this section , we cover the essentials of Query-by-Committee and Gaussian Mixture Models , as these constitute the backbone of the proposed acquisition functions . Active Learning The main idea of active learning ( AL ) is to actively choose a new data point to label and add to the current training data set ( Settles , 2009 ) . In the context of metamodeling , new data is often added sequentially , i.e. , one data point at a time ( Gramacy , 2020 ) , but in other applications , it can be beneficial to query batches of data instead ( Kirsch et al. , 2019 ) . The acquisition functions can be divided into model-based and model-free functions , where the former utilize information from the model and the latter do not ( O ’ Neill et al. , 2017 ) . Both types of functions seek to minimize the expected predictive loss of the model . Another approach is to minimize the number of possible models . Houlsby et al . ( 2011 ) divide the active learning acquisition functions into being based on either decision or information theory . Decision-based functions seek to minimize the expected predictive loss of the model in the hope of maximizing the performance on the test set . Information-theoretic-based functions instead try to reduce the number of possible models , for example , through the KL-divergence or Shannon entropy ( Houlsby et al. , 2011 ) . It is not straightforward to use information-theoretic acquisition functions . However , if you have access to the posterior of the model ’ s parameters , Houlsby et al . ( 2011 ) have derived the algorithm Bayesian Active Learning by Disagreement ( BALD ) , which can be applied in general . Generally , BALD seeks the data point that maximizes the decrease in the expected posterior entropy of the parameters . Query-by-Committee The Query-by-Committee ( QBC ) is a specific acquisition function that was originally proposed for classification tasks ( Seung et al. , 1992 ) . It aims to maximize the disagreement among the committee to get the highest information gain and minimize the version space , which is the set of model hypotheses aligned with the training data . The construction of the committee is the core component of QBC since it is the committee ’ s ability to accurately and diversely represent the version space that gives rise to informative disagreement criteria ( Settles , 2009 ) . QBC can also be applied for regression problems . Krogh & Vedelsby ( 1995 ) construct the members of the committee by random initializations of the weights in the neural networks . RayChaudhuri & Hamey ( 1995 ) apply bagging and train the members on different subsets of the data set . In general , QBC constructed by bagging has been used as a benchmark with mixed results ( Cai et al. , 2013 ; Wu , 2018 ; Wu et al. , 2019 ) . Burbidge et al . ( 2007 ) show that the less noise there is in the output , the better QBC is compared to random querying . They also highlight the fact that with a misspecified model , QBC might perform worse than random querying . None of these approaches explore the usage of MCMC samples of the posterior to construct a committee . To the best of our knowledge , we are the first to propose QBC based on model hypotheses drawn from the posterior . Gaussian Process as a Gaussian Mixture Model Mixture models have recently been applied in active learning for classification tasks . Iswanto ( 2021 ) propose to use Gaussian Mixture Models ( GMMs ) with active learning , where they design a specific acquisition function that queries the data point that maximizes the expected likelihood of the model . Zhao et al . ( 2020 ) use a mixture of GPs in active learning , where each component is fitted to a subset of the training set . The combination of GMMs and GPs have previously been explored for static data sets . Chen & Ren ( 2009 ) investigate regression tasks and apply bagging , where they repeatedly randomly sample data points from the training set to construct new subsets to get GPs fitted to different data . Among other combination rules , they combine the predictive posteriors of these GPs into GMMs . They obtain even better performance by weighting the models by the predictive uncertainty such that models with high uncertainty are given smaller weights , and vice versa . Instead of using bagging , we construct the multiple GPs by using the MCMC samples of the hyperparameters ’ joint posterior , and then obtain a natural weighting of the GPs : the GMM will consist of more GPs with hyperparameters close to the modes than hyperparameters far away . The formal procedure is given in section 4 . To the best of our knowledge , we are the first to combine GMMs and GPs in this manner . 3 GAUSSIAN PROCESSES . The Gaussian Processes ( GPs ) are the central models in this work . In this section , we give a brief overview of GPs before covering the Fully Bayesian GPs . For a thorough description of GPs , we refer to Williams & Rasmussen ( 2006 ) . Gaussian Processes A Gaussian Process ( GP ) is a stochastic function fully defined by a mean function m ( · ) and a covariance function ( often called a kernel ) k ( · , · ) . Given the data ( X , y ) = { xi , yi } Ni=1 , where yi is the corrupted observations of some latent function values f with Gaussian noise ε , i.e. , yi = fi + εi , εi ∈ N ( 0 , σ2ε ) , a GP is typically denoted as GP ( mf ( x ) , kf ( x , x′ ) ) . It is common practice to set the mean function equal to the zero-value vector and thus , the GP is fully determined by the kernel kf ( x , x′ ) . For short , we will denote the kernel Kθ , which explicitly states that the kernel is parameterized with some hyperparameters θ . Given the optimal hyperparameters , the predictive posterior for unknown test inputs x∗ is given by p ( f∗|θ̂ , y , X , X∗ ) = N ( µ∗ , Σ∗ ) with µ∗ = K∗θ ( Kθ + σ 2 εI ) −1 y and Σ∗ = K ? ? θ −K∗θ ( Kθ + σ 2 εI ) −1 K ? > θ ( 1 ) where K ? ? θ denotes the covariance matrix between the test inputs , and K ∗ θ denotes the covariance matrix between the test inputs and training inputs . Covariance matrix We use the canonical kernel automatic relevance determination ( ARD ) Radial-basis function ( RBF ) given by k ( x , x′ ) = exp ( −||x− x′||2/2 ` 2 ) where ` is a vector of length scales ` 1 , ... , ` d , one for each input dimension . Often the kernel is scaled by an output variance but here we fix it to one and solely focus on the two other hyperparameters : length scale and noise-term . The noise-term σ2ε is integrated into the kernel with an indicator variable by adding the term σ2εI { x=x′ } to the current kernel . Fully Bayesian Gaussian Processes ( FBGP ) A FBGP extends a GP by putting a prior over the hyperparameters θ ∼ p ( θ ) and approximate their full posteriors . The joint posterior is then given by p ( f , θ|y , X ) ∝ p ( y|f ) p ( f |θ , X ) p ( θ ) ( 2 ) and the predictive posterior for the test inputs X∗ is p ( f∗|y ) = ∫∫ p ( f∗|f , θ ) p ( f |θ , y ) p ( θ|y ) dfdθ ( 3 ) where the conditioning on X and X∗ have been omitted for brevity . The inner integral reduces to the predictive posterior given by a normal GP . However , the outer integral remains intractable and is approximated with MCMC inference as p ( f∗|y ) = ∫ p ( f∗|y , θ ) p ( θ|y ) dθ ' 1 M M∑ j=1 p ( f∗|y , θj ) , θj ∼ p ( θ|y ) ( 4 ) As well known in machine learning modeling , there is no free lunch . Adapting the hyperparameters of a FBGP is computationally expensive compared to the approach with GPs and maximum likelihood estimates . However in Bayesian optimization and active learning , the computational burden for querying a new data point will often be of magnitudes higher . Evaluation and Fitting of GPs A GP is typically fitted by maximizing the marginal likelihood with gradient descent , where the marginal likelihood can be computed analytically . For the FBGP , the inner integral is intractable and is therefore optimized with approximate inference using the selftuning method of Monte Carlo Markov Chain ( MCMC ) called the No-U-Turn-Sampler ( NUTS ) ( Hoffman & Gelman , 2014 ) . The MCMC inference makes the full posterior of the hyperparameters available , making it possible to extend the point estimates to a joint distribution . In the experiment , we apply acquisition functions that use the full posterior distribution ; however , since the experiments are designed to benchmark the acquisition functions , we always use the mode of the joint posterior for prediction and evaluation .

Summary: This paper introduces two new active-learning strategies for Gaussian process regression, based on a fully-Bayesian treatment of GPs. Instead of using fixed hyperparamers obtained by maximizing the marginal log-likelihood of the GP model, the authors suggest learning a posterior belief over GP hyperparameters, and make use of the full distribution to design more effective active-learning methods.  In practice, this results in two concrete proposals:  1) a strategy that is similar to query-by-committee, where the committee is formed of multiple GPs sampled from the hyper-posterior (B-QBC). 2) a variant that combines a notion of disagreement with the predictive uncertainty (QB-MGP)  These methods are then compared to others on 8 synthetic datasets.