page_title: Gaussian process
section_title: Usual covariance functions
hierarchical_section_title: Gaussian process / Covariance functions / Usual covariance functions
caption_attribution_description: English: Draws from the Gaussian process prior distribution with different kernels. Left is a squared exponential kernel. Middle is a Brownian Kernel. right is a quadratic kernel
context_page_description: In probability theory and statistics, a Gaussian process is a stochastic process, such that every finite collection of those random variables has a multivariate normal distribution, i.e. every finite linear combination of them is normally distributed. The distribution of a Gaussian process is the joint distribution of all those random variables, and as such, it is a distribution over functions with a continuous domain, e.g. time or space.
  A machine-learning algorithm that involves a Gaussian process uses lazy learning and a measure of the similarity between points to predict the value for an unseen point from training data. The prediction is not just an estimate for that point, but also has uncertainty information—it is a one-dimensional Gaussian distribution.
  For multi-output predictions, multivariate Gaussian processes
  are used, for which the multivariate Gaussian distribution is the marginal distribution at each point.
  For some kernel functions, matrix algebra can be used to calculate the predictions using the technique of kriging. When a parameterised kernel is used, optimisation software is typically used to fit a Gaussian process model.
context_section_description: There are a number of common covariance functions:
  Constant : 
  Linear: 
  white Gaussian noise: 
  Squared exponential: 
  Ornstein–Uhlenbeck: 
  Matérn: 
  Periodic: 
  Rational quadratic: 
  Here . The parameter  is the characteristic length-scale of the process (practically, "how close" two points  and  have to be to influence each other significantly),  is the Kronecker delta and  the standard deviation of the noise fluctuations. Moreover,  is the modified Bessel function of order  and  is the gamma function evaluated at . Importantly, a complicated covariance function can be defined as a linear combination of other simpler covariance functions in order to incorporate different insights about the data-set at hand.
  Clearly, the inferential results are dependent on the values of the hyperparameters  (e.g.  and ) defining the model's behaviour. A popular choice for  is to provide maximum a posteriori (MAP) estimates of it with some chosen prior. If the prior is very near uniform, this is the same as maximizing the marginal likelihood of the process; the  marginalization being done over the observed process values . This approach is also known as maximum likelihood II, evidence maximization, or empirical Bayes.
caption: The effect of choosing different kernels on the prior function distribution of the Gaussian process. Left is a squared exponential kernel. Middle is Brownian. Right is quadratic.
exif: {}