Sparse Gaussian Process Hyperparameters: Optimize or Integrate?
Abstract
The kernel function and its hyperparameters are the central model selection choice in a Gaussian process [Rasmussen and Williams, 2006].
Typically, the hyperparameters of the kernel are chosen by maximising the marginal likelihood, an approach known as Type-II maximum likelihood (ML-II).
However, ML-II does not account for hyperparameter uncertainty, and it is well-known that this can lead to severely biased estimates and an underestimation of predictive uncertainty.
While there are several works which employ a fully Bayesian characterisation of GPs, relatively few propose such approaches for the sparse GPs paradigm.
In this work we propose an algorithm for sparse Gaussian process regression which leverages MCMC to sample from the hyperparameter posterior within the variational inducing point framework of [Titsias, 2009].
This work is closely related to Hensman et al.
[2015b], but side-steps the need to sample the inducing points, thereby signiﬁcantly improving sampling efﬁciency in the Gaussian likelihood case.
We compare this scheme against natural baselines in literature along with stochastic variational GPs (SVGPs) along with an extensive computational analysis.
Introduction
Gaussian processes (GPs) are a prominent class of models for supervised learning which can quantify uncertainty and incorporate inductive biases in function space via the kernel function.
Hand-crafting a kernel function is a powerful way to incorporate prior knowledge.
In many instances not all properties of a kernel function can be speciﬁed from prior knowledge alone, and parameters are chosen via MLII.
However, deﬁning a complex kernel function with a large number of hyperparameters can make the marginal likelihood prone to multiple local optima and overﬁtting.
Further, several local optima may correspond to priors that do not sensibly model the data.
Weakly identiﬁed hyperparameters can manifest in ﬂat ridges in the marginal likelihood surface 1 making gradient based optimisation extremely sensitive to starting values [Warnes and Ripley, 1987].
Overall, the ML-II point estimates for the hyperparameters are subject to high variability and underestimate prediction uncertainty.
The problem of ridges in the marginal likelihood surface also does not necessarily go away as more observations are collected.
For example, if f 1 and f 2 are Brownian motions,  f 1 ( x/ ` ) is equal in distribution to {+p ↵ f 
+} ( x/ ↵` ) , which means observations do not provide any information about the product ` .
More generally, for a greater class of kernels, including the Mat´ern– 1 / 2 kernel,  f 1 ( x/ ` ) is equivalent to p ↵ f 2 ( x/ ↵` ) , which means that is not possible to consistently estimate the product ` from data, no matter how many observations are collected in a ﬁxed domain [Chapter 6, Stein, 1999].
This implies that one cannot estimate the individual hyperparameters (  , ` ) consistently.
It also motivates why there can be beneﬁts to estimating the hyperparameter posterior even in large data regimes, and ML-II may be insufﬁcient.
A more satisfactory treatment of hyperparameters involves placing a prior over the hyperparameters and performing Bayesian inference to compute a (hyper)posterior.
For large datasets, this motivates using scalable GP inference (e.g. sparse methods) in conjunction with Markov chain Monte Carlo (MCMC) for the hyperparametrs.
The Bayesian treatment of weakly identiﬁed hyperparameters may also be fraught with difﬁculties.
Gradient-based samplers like Hamiltonian Monte Carlo [Neal et al., 2011] and its variants have difﬁculty navigating regions of high curvature and ﬂat ridges where the gradient offers no information for transition [Betancourt, 2017].
This leads to over-concentration of samples from the ﬂat region.
(Usually, this can at least partially be rectiﬁed with informative priors.)
These pathologies are also typical of other hierarchical models [Betancourt and Girolami, 2015].
Figure 1 shows that the GP marginal likelihood surface can manifest these pathologies.
The evidence lower bound (ELBO) used in hyperparameter selection for variational sparse GPs relying on inducing points inherits similar, or even less favourable, characteristics to the exact marginal likelihood.
As a result, for weakly informative priors, gradient based samplers are susceptible to getting stuck at the boundary of these pathological regions hence biasing the sample estimates.
The effective sample size metric, used for diagnosing mixing in MCMC, is indicative of this behaviour when directly observing the phase space of the target distribution, but is infeasible in high-dimensions.
Historical justiﬁcation for ML-II (also called the evidence framework) comes from [MacKay, 1994] which highlighted several conditions for ML-II to yield reasonable estimates for the hyperparameters.
Crucially, the evidence is unlikely to manifest multiple local optima for a model well-matched to the data and with a high signal-to-noise ratio.
Transferring this insight to the Gaussian process paradigm we show how the evidence can have a signiﬁcant tail or no well-deﬁned maximum in settings with a low signal-to-noise ratio which arises with high aleatoric uncertainty or sparse data, frequently both.
In these settings, a point estimate may not adequately summarise the hyperparameter posterior and the beneﬁts of marginalisation stand out.
The situation is only exacerbated in high-dimensions (i.e. when there are many hyperparameters) where increasingly more volume of the posterior is captured in a thin shell making the density peak extremely unrepresentative of the posterior.
Unlike the likelihood of parameters, the marginal likelihood inherently contains a trade-off between the data-ﬁt and complexity penalty term.
This is one of the main properties that makes the marginal likelihood objective a viable choice for model selection [Rasmussen and Williams, 2006].
For example, for the Gaussian process regression model,
[Equation]
the marginal likelihood takes the form,
[Equation]
data ﬁt term
[Equation] [Equation]
complexity penalty
[Equation]
where c is a constant, p ( y | f ) denotes the data likelihood and ✓ denotes kernel hyperparameters.
This trade-off is a well-established idea which embodies the automatic Occam’s razor effect [Rasmussen and Ghahramani, 2001] where models well-suited to the data are automatically selected just by using the marginal likelihood objective.
This may seem to contradict earlier claims regarding overﬁtting, but by shying away from dealing with the hyperparameter posterior we risk overﬁtting even with the marginal likelihood approach.
In other words, the evidence framework subdues the overﬁtting effect induced by the canonical maximum likelihood approach which does not have any complexity penalty term.
The parameters in the canonical approach are free to ﬁt the data as well as possible making them prone to overﬁtting and poor generalisation.
Overparameterized kernels based on neural networks like deep kernel models [Wilson et al., 2016] are well known to exacerbate overﬁtting [Ober et al., 2021].
The main motivation for this work is to highlight that fully Bayesian schemes in sparse Gaussian process models are practically beneﬁcial.
While several works in the literature employ fully Bayesian scheme of integrating out the hyperparameters (see table 1), in this work we attempt to analyse them in an orthogonal direction, focusing on comparison with the evidence framework and other benchmarks by extending the main sparse variational formulation in the literature Titsias [2009].
We present a generalised inference scheme for fully Bayesian GP regression and counteract some of the computational cost of sampling both inducing variables and hyperparameters by deriving a doubly collapsed bound which selects the optimal distribution over the inducing points analytically and targets the kernel hyperparameters with HMC.
Related Work
[Table]
Fully Bayesian Gaussian processes have been used by several authors spawning several variants.
In early accounts, Neal [1998], Williams and Rasmussen [1996] explore the integration over covariance hyperparameters using HMC in the regression setting.
Barber and Williams [1997] extend this to the classiﬁcation setting using HMC for sampling in the hyperparameter space and Laplace approximation for the integrand over function values.
Murray and Adams [2010a] and Filippone et al.
[2013] focused on MCMC schemes to sample covariance hyperparameters in conjunction with latent function values, mainly mitigating the coupling effect through reparameterisation.
Hensman et al.
[2015b] considered joint sampling of inducing variables and hyperparameters from the optimal variational posterior distribution while [Bui et al., 2018] consider inference schemes for fully Bayesian sparse GPs in a streaming setting.
More recently, Rossi et al.
[2021] studied fully Bayesian sparse GPs using
SG-HMC.
Rossi et al.
[2021] modify the generative model by adding a prior over the inducing inputs, and peform inference using SG-HMC over the joint ( Z , u , ✓ ) space.
We list the most recent works in
Table 1.
Background
Let f ⇠ GP (0 , k ✓ ) be a Gaussian process prior with kernel function k ✓ depending on hyperparameters ✓ .
We are given noisy observations y
=
( y n ) Nn =1 ✓ R of f = ( f ( x n ))
Nn =1 at input data X = ( x n ) Nn =1 ✓ R D .
We consider a Gaussian likelihood which factorises over the data, p ( y | f ) =
Q
Nn =1 N ( y n | f n ,  2 ) .
We wish to compute the posterior p ( f | y , ✓ ) .
In this section, we recapitulate the canonical inducing variable approximation of p ( f | y , ✓ ) by Titsias [2009] and its extension to a Bayesian treatment of the hyperparameters.
3.1 Sparse variational inference in Gaussian processes
Following Titsias [2009], we consider a set of inducing variables u
= { f ( z m ) } Mm =1 ✓ R at inducing inputs Z = { z m } Mm =1 , z m 2 R d .
The complete generative model can then be factored as,
[Equation]
We approximate the posterior p ( f, u | y , ✓ ) with a variational distribution:
[Equation]
where q ( u ) is chosen to minimise the Kullback–Leibler divergence KL( q ( f | ✓ ) k p ( f | y , ✓ )) .
Minimising this KL divergence corresponds to maximising the evidence lower bound [Matthews et al., 2016], henceforth called the ELBO:
[Equation]
Because the ELBO still depends on q ( u ) , this bound is called uncollapsed .
Hensman et al.
[2013, 2015a] let q ( u ) be a Gaussian, which is optimal if the likelihood is Gaussian [Titsias, 2009], approximate the expectation using Monte Carlo, and maximise the ELBO using stochastic optimisation.
On the other hand, if the likelihood is Gaussian, Titsias [2009] computes the optimal form for q ( u ) directly:
[Equation]
Plugging q ⇤ ( u | ✓ ) back into equation (5), the resulting bound is called collapsed , because it now only depends on ✓ and Z .
The collapsed bound, denoted L ✓ ,Z , is the objective that Titsias [2009] proposes:
[Equation]
where K nn is the prior covariance matrix of f , K mm is the prior covariance matrix over u and K nm is crosss-covariance matrix formed by f and u .
Using the collapsed bound, approximate ML-II consists of ﬁnding,
[Equation]
Predictions at new functions values can be made in O ( M 2 ) after an initial cost of O ( NM 2 ) .
Under certain assumptions on the data generating process, even when M ⌧ N , the approximate posterior closely resembles the posterior, and equation (7) is a provably accurate approximation to equation (2)
[Burt et al., 2020].
3.2 Bayesian treatment of hyperparameters and sparse methods
The extension of the sparse variational framework to a Bayesian treatment of the hyperparameters has been previously considered by Hensman et al.
[2015b].
Extend the generative model with a prior p ( ✓ ) over the hyperparameters ✓ , and let the variational approximation of the posterior p ( f, u , ✓ | y ) be q ( f, u , ✓ )
= p ( f | u , ✓ ) q ( u , ✓ ) .
The analogue of equation (5) is
[Equation]
and the optimal form for q ( u , ✓ ) can again be determined:
[Equation]
[Equation]
The distribution q ⇤ ( u , ✓ ) does not have a closed form, and for general likelihoods, Hensman et al.
[2015b] propose to approximate the expectation in (10) with quadrature and to sample from q ⇤ ( u , ✓ ) using HMC.
While this approach is quite general, in the case of Gaussian regression, it vastly increases the dimensionality of the state space over which HMC must be run relative to HMC in GPR, since the u are sampled in addition to the ✓ .
This increases the cost of the procedure, and impacts the success of the sampler.
An alternative approach to approximately inferring hyperparameters is to assume a parametric form for q ( u , ✓ ) and maximise equation (9) with respect to the variational parameters.
Bui et al.
[Table]
took such an approach [Table] assuming that q ( u , ✓ ) =
q ( u ) q ( ✓ ) [Table] with both distributions Gaussian.
Similar approaches have been applied to variational inference in state-space modelling, sometimes leveraging the optimal form of q ( u | ✓ ) discussed earlier
[Table]
The QUALITY column in Table 2 indicates the ability of the method to faithfully represent the hyperparameter posterior.
If VI is run to convergence, a potentially signiﬁcant amount of error will be incurred by the Gaussian approximation to the non-Gaussian posterior over the hyperparameters (red).
On the opposite extreme, if no sparsity assumption is made, MCMC over the hyperparameters without sparse approximations is asymptotically consistent (green).
The inducing point approximations combined with MCMC lie somewhere in-between these methods yellow).
3.3 Making predictions
The predictive posterior distribution for unknown test inputs X ⇤ integrates over the joint posterior,
[Equation]
where we have suppressed the conditioning over inputs X, X ⇤ for brevity.
The inner integral simpliﬁes to R p ( f ⇤ | f , u , ✓ ) p [Equation] f | u , ✓ )
d f = p ( f ⇤ | u , ✓ ) .
We discuss the predictive posterior in such models in section 4.3.
Fully Bayesian SGPR with HMC: Doubly collapsed formulation
In the previous section, we observed that a major drawback of the approach taken in Hensman et al.
[2015b] is the need to sample u , which for high-dimensional inputs or in cases where many inducing points are needed could introduce thousands of additional variables to sample.
In this section, we leverage the optimal form of q ( u | ✓ ) derived in Titsias [2009] to alleviate this sampling problem.
4.1 Collapsing the evidence lower bound (again)
We ﬁrst derive the lower bound for this formulation and provide pseudo-code for the algorithm in
Algorithm 1.
Following the usual derivation of the ELBO,
[Equation]
[Equation]
where log p ( y | ✓ )  L ✓ ,Z with L ✓ ,Z deﬁned in equation (7), and where we assign M ✓ ,Z = e L ✓ ,Z .
4.1.1 Deriving q ⇤ ( ✓ )
We can interpret L ⇤ Z ( q ( ✓ )) as a negative KL divergence as long as we account for a normalisation constant C Z = R M ✓ ,Z p ( ✓ ) d ✓ for the un-normalised numerator M ✓ ,Z p ( ✓ ) .
Hence, we can re-write
L ⇤ Z ( q ( ✓ )) as,
[Equation]
where q ⇤ ( ✓ ) =
M ✓ ,Z p ( ✓ ) /C
Z .
By inspecting equation (14), we observe that the optimal variational distribution over ✓ is given by q ⇤ ( ✓ ) .
2 Crucially, by sampling directly from q ⇤ ( ✓ ) using MCMC we eliminate the need to sample the variables u .
By evaluating L ⇤ Z at q ⇤ ( ✓ ) , we ﬁnd the doubly collapsed
ELBO L ⇤⇤ Z :=
L ⇤ Z ( q ⇤ ( ✓ )) =
log C Z .
Although the value of L ⇤⇤ Z is computationally intractable, given samples ( ✓ j ) Jj =1 from q ⇤ ( ✓ ) , gradients of L ⇤⇤ Z with respect to Z can be estimated using the stochastic estimate of the canonical ELBO equation (7): using the chain rule,
[Equation]
where  q L ⇤ Z ( q ) is the functional derivative of L ⇤ Z ( q ) with respect to q , which is zero at q = q ⇤ ( ✓ ) , because q ⇤ optimises L ⇤ Z (it is a critical point, so the derivative is zero).
Further, the partial derivative of L ⇤ Z with respect to Z concerns just the ﬁrst term of the LHS of equation (13) as the KL term
KL( q ( ✓ ) k p ( ✓ )) is independent of Z .
4.2 Performing approximate inference
We deploy HMC to (approximately) sample from the optimal variational posterior q ⇤ ( ✓ ) along with optimising the inducing inputs Z in a hybrid scheme.
We alternate between the two steps allocating longer intervals for optimising Z for every HMC sampling run for the hyperparameters.
We note that this hybrid scheme is much more computationally efﬁcient than sampling u and ✓ jointly where one has to tackle the coupling between inducing variables and hyperparameters in joint space.
Further, joint sampling is only feasible for moderate number of inducing variables while this scheme can scale to much larger datasets as the efﬁciency of sampling in the hyperparameter space is only dependent on the dimensionality of the hyperparameter space rather than the number of inducing variables.
The entire inference scheme is summarized in Algorithm 1.
The warm-start strategy (of optimizing both ( Z, ✓ ) jointly for a few gradient steps) is used to ﬁnd a good region for the sampler to initialise ✓ .
1: Input: ELBO objective L ✓ ,Z :=
L ( ✓ , Z ) (equation (7)), gradient based optimiser optim() 2: 3: procedure W ARM -S TART 4: for ﬁxed number of steps do 5:
Gradient step: Z, ✓  optim ( L ( ✓ , Z ) ) 6: return initial values Z init and ✓ init 7:8: procedure T RAIN 9: ## Initialisation protocol 10: I Initialise L ✓ ,Z at warm-start values L ( ✓ init , Z init ) , lets call this ˆ L 11: I Freeze kernel hyperparameters in the ELBO objective by setting requires grad =
False .
12: 13: while not converged do 14: for t = 1 . . .
T do ## start of training loop 15:16: • Gradient step: Z opt  optim ( ˆ L ) (## equation (15) shows the validity of taking the derivative of the stochastic ELBO) 17: if t mod L == 0 then 18: (## For every L gradient steps) 19:20: j ) HMC
(where is Hamiltonian) 22: where p denotes the mean momentum variable in phase space with 23: the same dimensionality as ✓ .
P Jj = j
) 25: return } Jj =
4.3 Predictive posterior
It is ultimately the posterior predictive (PP) distribution that is of interest rather than point estimates or the hyperparameter posterior.
The Bayesian sparse GP predictive posterior entails integrating out the posterior over inducing variables u and hyperparameters ✓ .
Once we have performed inference, we can approximate this directly, p ( f ⇤ | y )
[Equation]
where q ⇤ ( u | ✓ , y ) =
N ( m ⇤ , S ⇤ ) is the optimal Gaussian variational distribution and
A =
K ⇤ m K  1 mm under the SGPR scheme (Section 4) is available in closed form with m ⇤ =
2 ( K mm +   2 K mn K nm ) and S ⇤ =
K mm ( K mm +   2 K mn K nm )  1 K mm .
In either case, the internal integral with respect to the inducing variables is analytic and the outer integral can be estimated using the samples collected from HMC to perform Monte Carlo estimation.
This yields a mixture of
Gaussians,
[Equation]
where J samples are (approximately) drawn from q ⇤ ( ✓ ) via HMC.
The distribution inside the summation is the Gaussian posterior predictive distribution for ﬁxed hyperparameters with identical mixing proportions.
The compute cost for the predictive posterior scales the sparse GPR cost linearly in the number of samples.
The Monte Carlo approximation costs O ( JNM 2 ) for M inducing points and J hyperparameter samples.
Experiments
In the previous section, we discussed a hybrid scheme which leverages MCMC within the variational sparse inducing variable formulation leading to fully Bayesian sparse Gaussian processes.
In the experiments we demonstrate the feasibility of this scheme relative to several benchmarks and assess regression performance on a 1-dimensional illustrative example and a range of other datasets.
We also compare with exact GPs where the cost of using a gradient based sampler is prohibitive for even moderately large datasets requiring several inversions of the full covariance matrix.
We show that using the doubly collapsed scheme proposed in this work is a much more attractive alternative for large datasets as compared to direct HMC in GPR, and sampling is more efﬁcient than in the uncollapsed bound used in Hensman et al.
[2015b].
We henceforth refer to benchmark methods as follows: SGPR + HMC refers to Bayesian GPs with doubly collapsed variational inference with NUTS, as described in section 3 (compatible with Gaussian likelihoods); we benchmark this model against sparse GPs ( SGPR )
[Titsias, 2009] and Stochastic Variational GPs ( SVGP ) both using approximate ML-II [Hensman et al., 2015a].
Additionally, we consider the sparse, joint sampling inference scheme proposed in [Hensman et al., 2015b] which gives a natural benchmark.
We call this model JointHMC .
The FBGP method extends the Bayesian treatment to the inducing locations similar to Rossi et al.
[2021]; we use NUTS to sample from the joint posterior over ( Z, ✓ ) .
It is not a direct comparison to Rossi et al.
[2021] as the latter explores free-form sampling of u along with ( Z, ✓ ) while we work with the collapsed bound incorporating the optimal Gaussian variational distribution q ⇤ ( u ) .
We also present analysis where we ﬁx inducing point locations at a random subset of the training data (as opposed to interleaving as per Algorithm 1) and only learn hyperparameters using NUTS.
We provide several details about the experimental set-up in the supplementary.
5.1 One dimensional synthetic data
[Table]
We sample noisy observations from f ( x ) = sin(3 x ) + 0 . [-
-]3 cos( ⇡ x ) with the constraint ( x <  2) and ( x > 2) .
Figure 2 shows the results for
SGPR along with the fully Bayesian schemes.
We keep data split and noise identical across the three models to facilitate a comparison.
While there is signiﬁcant data to identify the hyperparameters we notice that the models mainly differ in their extrapolation abilities away from the training data.
SGPR with ML-II overﬁts to the training data and recovers a low lengthscale, low noise solution while the SGPR + HMC scheme recovers a more moderate ﬁt and performs signiﬁcantly better in terms of RMSE and NLPD on unseen data Table 3.
We note that the JointHMC scheme which samples both ( u , ✓ ) overﬁts in the central missing data region.
We use M =inducing locations across all methods which are optimised according to the protocol of each method and recover a very similar spatial distribution.
[Table]
We compare our approach across methods on 5 standard small to medium-sized UCI benchmark datasets.
Following common practice, we use a 20 % randomly selected held out test-set
[Rossi et al., 2021, Havasi et al., 2018] and scale the inputs and outputs to zero mean and unit standard deviation within the training set (we restore the output scaling for evaluation) [Salimbeni and Deisenroth, 2017].
While we could use any kernel, we choose the RBF-ARD kernel with a lengthscale for each dimension.
For consistency we initialise all the inducing locations ( Z ) identically across the methods, i.e. by using the same random subset of training data split.
We note that adapting the inducing locations brings serious gains in prediction performance versus keeping them ﬁxed (Figure 3).
Further, the
JointHMC scheme underperforms SGPR (with ML-II) and SGPR + HMC.
This is not surprising given that the JointHMC bound equation (10) does not incorporate the optimal setting for q ( u ) and was originally motivated by the need for a fully Bayesian scheme for generalised likelihoods.
The method SGPR + HMC signiﬁcantly improves upon JointHMC in the speciﬁc Gaussian likelihood case.
Sensitivity to M : We benchmark SGPR+HMC and JointHMC on the Elevator dataset ( N = 16599 , D = 18 ) which demands a larger M .
SGPR+HMC outperforms JointHMC for this dataset across different M but the advantage is more pronounced at smaller M .
Further, our method took 1248 vs. 2109 wall clock sec.
for the joint scheme for the same number of hyperparameter samples and 500 inducing points.
5.3 Ablation study
In order to understand the efﬁcacy of Algorithm 1 we conduct an ablation study where we perform inference in the same manner, but keeping inducing locations ﬁxed.
Algorithmically, this implies that we don’t need to compute the stochastic ELBO ˆ L and just conduct a single sampling run for the hyperparameters.
The results across 10 splits are summarised in Table 5.
[Table]
5.4 Runtimes
While it is possible to train exact GPR with HMC for datasets of this size (in terms of N ) it is important to look at the trade-off in terms of compute cost.
In Figure 4 we record the average number of wall clock seconds to draw 500 samples under each method.
The cost of sampling is O ( N ) for
SGPR + HMC
but O ( N 3 ) for Exact GPR + HMC.
JointHMC deals with a higher dimensional phase space ( u , ✓ ) hence requires more tuning.
We don’t include tuning time for a fair comparison.
For further context we report the total training run-time for our scheme alongside ML-II, GPR + HMC and FBGP in Table 6.
The hybrid scheme we propose is signiﬁcantly cheaper to canonical alternatives with virtually no degradation in predictive performance
[Table]
[Table]
[Table]
Discussion
The evidence framework continues to be the pre-dominant method for training Gaussian processes since their inception into modern machine learning [Rasmussen and Williams, 2006].
While the marginal likelihood is a compelling model selection objective as it offers an inherent trade-off between data-ﬁt and complexity
[Table]
it is susceptible to overﬁtting and other pathologies leading to biased inference.
This work builds on existing methods that combine sparse Gaussian process regression based on inducing variables with Bayesian hyperparameter inference.
Bayesian hyperparameter inference in GPs is however intractable and one has to consider balancing between objectives of computational cost, prediction accuracy and robustness of uncertainty intervals.
While in straightforward conditions the fully Bayesian approach might be counter-productive
[Table]
most real-world applications of GPs rely on engineering sophisticated hand-crafted kernels involving many hyperparameters where there risk of overﬁtting is pronounced and further, harder to detect.
A more robust solution is to incorporate prediction intervals that reﬂect these uncertainties in the model choice.
Studying full Bayesian inference in more sophisticated GP models like deep [Damianou and
Lawrence, 2013], warped [Snelson et al., 2004] and convolutional GPs [Van der
Wilk et al., 2017] will offer greater insight to this question and is an imminent direction of future work.
Acknowledgements
This research was conducted while WPB and DRB were students at the University of Cambridge.
During that time, WPB was supported by the Engineering and Physical Research Council (studentship number 10436152).
VL acknowledges funding from the Qualcomm Innovation Fellowship (Europe).
[-1 + 2 (cid:80) Tt =
-]