# Efficient Time-Series Clustering through Sparse Gaussian Modeling
## 3 Gaussian Process Regression for Time Series
### Gaussian Processes for Time Series  
At a conceptual level, a _Gaussian Process_ extends the notion of multivariate Gaussian distributions to infinite dimensions (and thus, to distributions over functions). Formally, a Gaussian Process is a stochastic process \((Z_{1},\ldots,Z_{t},\ldots)\) such that the joint distribution of every finite collection \((Z_{t_{1}},\ldots,Z_{t_{k}})\) of its random variables is a _multivariate Gaussian distribution_ ([14], Definition 2.1).  
For this work, it is convenient to regard a Gaussian Process model as a probability distribution over continuous functions (or over time series). For simplicity and since in this work we mostly deal with real-valued time series, where the observed data points \(x_{i}\in\mathbb{R}\), we restrict our attention to regression over functions that map points in time to real data points. Starting with the special case where a time series is evaluated in a fixed number \(n\) of points in time, for any \(n\)-dimensional vectors \(\vec{x}=(x_{1},\ldots,x_{n})\) and \(\vec{t}=(t_{1},\ldots,t_{n})\in\mathbb{R}^{n}\), a \(n\)-variate Gaussian distribution \(\mathcal{N}(m(\vec{t}),k(\vec{t},\vec{t}\,))\) determines the probability that the observed data points \((x_{1},\ldots,x_{n})\) at times \((t_{1},\ldots,t_{n})\) are drawn from a \(n\)-variate Gaussian distribution with mean function \(m:\mathbb{R}\to\mathbb{R}\) and covariance function (a.k.a. _kernel_ function) \(k:\mathbb{R}\times\mathbb{R}\to\mathbb{R}\). In fact, given a time series \(X=(\vec{x},\vec{t}\,)\), we want to compute a \(m(\vec{t}\,)=(m(t_{1}),\ldots,m(t_{n}))\) and positive semidefinite covariance matrix \(K=(k(t_{i},t_{j}))_{i,j\in[n]}\)) maximizing the probability that the observed data point vector \(\vec{x}\) is drawn the \(n\)-variate Gaussian distribution \(\mathcal{N}(m(\vec{t}\,),k(\vec{t}\,))\).  
The notion of a Gaussian Process naturally extends the idea of a \(n\)-variate Gaussian distribution to arbitrary dimensions, which allows us to regress over continuous functions and time series. A Gaussian process is defined by a _mean function_\(m:\mathbb{R}\to\mathbb{R}\) and a _kernel function_\(k:\mathbb{R}\times\mathbb{R}\to\mathbb{R}\). The mean function \(m(\cdot)\) determines the expected value \(m(t)\) of a data point \(x_{t}\in\mathbb{R}\) at any time \(t\). The kernel function \(k(\cdot\,,\,)\) quantifies the correlation \(k(t,t^{\prime})\) between the observed data points \(x_{t}\) and \(x_{t^{\prime}}\) at any two times \(t\) and \(t^{\prime}\). We let \(f\sim\mathcal{GP}(m,k)\) denote that the time series described by the function \(f:\mathbb{R}\to\mathbb{R}\) (i.e., the values of the data points are \(f(t)\) at all times \(t\)) follows a Gaussian Process \(\mathcal{GP}\) with mean function \(m\) and kernel function \(k\).  
The mean function \(m\) determines the average value of data points over time and is usually normalized to 0 over the entire time horizon. The kernel function \(k\) determines the shape of the time series modeled by the Gaussian Process, in the sense that if two points in time \(t\) and \(t^{\prime}\) are highly correlated (e.g., because \(t\) and \(t^{\prime}\) are neighboring points in time, or because we expect that the observed data points at \(t\) and \(t^{\prime}\) should be close to each other), the kernel function should favor time series \(f\) with similar values \(f(t)\) and \(f(t^{\prime})\).