Repository: GPflow/GPflow

Issue:
GPR Posterior Calculations incomplete

# Bug

When you call the `posterior()` method, `posterior.alpha` should be calculated, but currently[ it just sets `posterior.alpha` equal to y](https://github.com/GPflow/GPflow/blob/d088c2c4583df606a6f0ba79eea78d27c6210f7c/gpflow/posteriors.py#L264).

## To reproduce

**Minimal, reproducible example**
<!-- We need to be able to reproduce the bug by simply copy and pasting your code -->
```python
gpr_model = gpflow.models.GPR( data=(train_x, train_y), kernel=kernel)
posterior = gpr_model.posterior()
assert  all( posterior.alpha.numpy() == train_y )
```

## Expected behavior

Obviously there are calculation steps missing in the `posterior` method.

## System information

* GPflow version: 2.3.0





Discussion:

Resolving patch:
diff --git a/doc/source/notebooks/advanced/fast_predictions.pct.py b/doc/source/notebooks/advanced/fast_predictions.pct.py
--- a/doc/source/notebooks/advanced/fast_predictions.pct.py
+++ b/doc/source/notebooks/advanced/fast_predictions.pct.py
@@ -60,12 +60,10 @@
 # + [markdown] id="GX1U-fYPKPrt"
 # ## What can be cached?
 #
-# We cache two separate values: $\alpha$ and $Q^{-1}$. These correspond to the parts of the mean and covariance functions respectively which do not depend upon the test points. Specifically, in the case of the GPR model these are:
+# We cache two separate values: $\alpha$ and $Q^{-1}$. These correspond to the parts of the mean and covariance functions respectively which do not depend upon the test points. In the case of the GPR these are the same value:
 # \begin{equation*}
-#     \alpha = [K_{mm} + \sigma^2I]^{-1}\mathbf{y}\\ Q^{-1} = [K_{mm} + \sigma^2I]^{-1}
+#     \alpha = Q^{-1} = [K_{mm} + \sigma^2I]^{-1}
 # \end{equation*}
-# _(note in practice, we cache the cholesky decomposition of Q to take advantage of the 'base_conditional_with_lm' utility function)_
-#
 # in the case of the VGP and SVGP model these are:
 # \begin{equation*}
 #     \alpha = K_{uu}^{-1}\mathbf{u}\\ Q^{-1} = K_{uu}^{-1}
@@ -81,7 +79,6 @@
 # +
 import gpflow
 import numpy as np
-import tensorflow as tf
 
 # Create some data
 X = np.linspace(-1.1, 1.1, 1000)[:, None]
diff --git a/gpflow/posteriors.py b/gpflow/posteriors.py
--- a/gpflow/posteriors.py
+++ b/gpflow/posteriors.py
@@ -245,30 +245,34 @@ def _conditional_with_precompute(
         Kmn = self.kernel(self.X_data, Xnew)
         Knn = self.kernel(Xnew, full_cov=full_cov)
 
-        return base_conditional_with_lm(Kmn, self.Qinv, Knn, self.alpha, full_cov=full_cov)
+        err = self.Y_data - self.mean_function(self.X_data)  # type: ignore
 
-    def _precompute(self) -> Tuple[tf.Tensor, tf.Tensor]:
+        Knm = tf.transpose(Kmn)
+        mean = Knm @ self.alpha @ err
+        cov = Knn - Knm @ self.Qinv @ Kmn
 
-        """
-        Precomputes the cholesky decomposition of Kmm_plus_s for later reuse we will call
-        base_conditional_with_lm implementation ('Qinv' in the Abstract Posterior class). We also
-        precompute the less compute intensive error term ('alpha' in the Abstract Posterior class)
-        """
+        # The GPR model only has a single latent GP.
+        if full_cov:
+            cov = cov[None, ...]
+        else:
+            cov = tf.linalg.diag_part(cov)[:, None]
 
+        return mean, cov
+
+    def _precompute(self) -> Tuple[tf.Tensor, tf.Tensor]:
         Kmm = self.kernel(self.X_data)
         Kmm_plus_s = add_noise_cov(Kmm, self.likelihood_variance)
 
-        # obtain the cholesky decomposition of Kmm_plus_s
         Lm = tf.linalg.cholesky(Kmm_plus_s)
+        Kmm_plus_s_inv = tf.linalg.cholesky_solve(Lm, tf.eye(self.X_data.shape[0], dtype=Lm.dtype))
 
-        alpha = self.Y_data - self.mean_function(self.X_data)  # type: ignore
         tf.debugging.assert_shapes(
             [
-                (Lm, ["M", "M"]),
+                (Kmm_plus_s_inv, ["M", "M"]),
                 (Kmm, ["M", "M"]),
             ]
         )
-        return alpha, Lm
+        return Kmm_plus_s_inv, Kmm_plus_s_inv
 
     def _conditional_fused(
         self, Xnew, full_cov: bool = False, full_output_cov: bool = False
@@ -320,13 +324,6 @@ def __init__(
             self.update_cache(precompute_cache)
 
     def _precompute(self) -> Tuple[tf.Tensor, tf.Tensor]:
-
-        """
-        Precomputes the cholesky decomposition of Kmm_plus_s for later reuse we will call
-        base_conditional_with_lm implementation ('Qinv' in the Abstract Posterior class). We also
-        precompute the less compute intensive error term ('alpha' in the Abstract Posterior class)
-        """
-
         # taken directly from the deprecated SGPR implementation
         num_inducing = self.inducing_variable.num_inducing
         err = self.Y_data - self.mean_function(self.X_data)  # type:ignore


Test patch: