# A Gaussian Process Method with Uncertainty Quantification for Air Quality Monitoring
## 3 Uncertainty Quantification in Gaussian Processes
### Uncertainty in Measurements
Similarly, the partial derivative of Equation (7) with respect to \(\sigma_{n}^{2}\) is  
\[\frac{\partial\text{cov}(\mathbf{f}_{*})}{\partial\sigma_{n}^{2}}=(\mathbf{K }_{*}\mathbf{P})(\mathbf{D}_{\mathbf{K}}+\sigma_{n}^{2}\mathbf{I})^{-2}( \mathbf{K}_{*}\mathbf{P})^{T}=\sum_{i=1}^{n}\Lambda_{i}^{-1}\mathbf{\bar{p}}_ {i}\mathbf{\bar{p}}_{i}^{T}, \tag{19}\]  
where we denote the \(m\times n\) dimension matrix \(\mathbf{K}_{*}\mathbf{P}\) as  
\[\mathbf{K}_{*}\mathbf{P}=[\mathbf{\barHowever, the log marginal likelihood could be non-convex with respect to the hyperparameters, which implies that the optimisation may not converge to the global maxima [22]. A common solution dealing with it is to sample multiple starting points from a prior distribution, then choose the best set of hyperparameters according to the optima of the log marginal likelihood. Let's assume \(\mathbf{\theta}=\{\theta_{1},\theta_{2},\cdots,\theta_{s}\}\) being the hyperparameter set and \(\theta_{s}\) denoting the \(s\)-th of them, then the derivative of \(\log p(\mathbf{y}|\mathbf{X})\) with respect to \(\theta_{s}\) is  
\[\frac{\partial}{\partial\theta_{s}}\log p(\mathbf{y}|\mathbf{X},\mathbf{\theta})= \frac{1}{2}\text{tr}\Big{(}(\mathbf{\alpha}\mathbf{\alpha}^{T}-(\mathbf{K}+\sigma_{n}^ {2}\mathbf{I})^{-1})\frac{\partial(\mathbf{K}+\sigma_{n}^{2}\mathbf{I})}{ \partial\theta_{s}}\Big{)}, \tag{23}\]  
where \(\mathbf{\alpha}=(\mathbf{K}+\sigma_{n}^{2}\mathbf{I})^{-1}\mathbf{y}\), and \(\text{tr}(\cdot)\) denotes the trace of a matrix. The derivative in Equation (23) is often multimodal and that is why a fare few initialisations are used when conducting convex optimisation. [PERSON] et al. show that the optimisation process with various initialisations can result in different hyperparameters [22]. Nevertheless, the performance (prediction accuracy) with regard to the standardised root mean square error does not change much. However, the authors do not show how the variation of hyperparameters affects the prediction uncertainty [22]. An intuitive explanation to the fact of different hyperparameters resulting with similar predictions is that the prediction shown in Equation (6) is non-monotonic itself with respect to hyperparameters. To demonstrate this, a direct way is to see how the derivative of (6) with respect to any hyperparameter \(\theta_{s}\in\mathbf{\theta}\) changes, and ultimately how it affects the prediction accuracy and uncertainty. The derivatives of \(\mathbf{\tilde{f}}_{s}\) and \(\text{cov}(\mathbf{f}_{*})\) of \(\theta_{s}\) are as below  
\[\frac{\partial\mathbf{\tilde{f}}_{s}}{\partial\theta_{s}}=\Big{(}\mathbf{K}_{* }\frac{\partial(\mathbf{K}+\sigma_{n}^{2}\mathbf{I})^{-1}}{\partial\theta_{s }}+\frac{\partial\mathbf{K}_{*}}{\partial\theta_{s}}(\mathbf{K}+\sigma_{n}^{ 2}\mathbf{I})^{-1}\Big{)}\mathbf{y}. \tag{24}\]  
We can see that Equations (24) and (25) are both involved with calculating \((\mathbf{K}+\sigma_{n}^{2}\mathbf{I})^{-1}\), which becomes enormously complex when the dimension increases. In this paper, we focus on investigating how hyperparameters affect the predictive accuracy and uncertainty in general. Therefore, we use the Neumann series to approximate the inverse [21].