Repository: GPflow/GPflow

Issue:
SwitchedLikelihood fails in conjunction with likelihoods relying on the quadrature fallback
Are there any tips and trick to using the Student T likelihood? Below is an example of some data that I have... similar to the gpflow example of a coregionalized model with a student T.  When I run the code below, I get this error.

```   File "/opt/conda/lib/python3.6/site-packages/gpflow-1.3.0-py3.6.egg/gpflow/quadrature.py", line 187, in eval_func
    feval = f(*Xs, **Ys)  # f should be elementwise: return shape N x H**Din
  File "/opt/conda/lib/python3.6/site-packages/gpflow-1.3.0-py3.6.egg/gpflow/decors.py", line 67, in tensor_mode_wrapper
    result = method(obj, *args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/gpflow-1.3.0-py3.6.egg/gpflow/likelihoods.py", line 217, in logp
    return logdensities.student_t(Y, F, self.scale, self.df)
  File "/opt/conda/lib/python3.6/site-packages/gpflow-1.3.0-py3.6.egg/gpflow/logdensities.py", line 59, in student_t
    tf.log(1. + (1. / df) * (tf.square((x - mean) / scale)))
```

maybe a convergence error...? I have tried playing around with the jitter parameter and the degrees of freedom, but to no avail. Any ideas?

Thanks!

 Below is a minimal bug-reproducing example
```
import gpflow as gpflow
import numpy as np

idx = np.array([0]*12 + [1]*15 + [2]*36)
X_aug = np.c_[np.random.randn(12+15+36, 1), idx]
Y_aug = np.c_[np.random.randn(12+15+36, 1), idx]

k1 = gpflow.kernels.Matern32(1, variance=.1, active_dims=[0]) +gpflow.kernels.White(1)
coreg = gpflow.kernels.Coregion(1, output_dim=3, rank=3, active_dims=[1])
kern = k1 * coreg

# build a variational model. This likelihood switches between Student-T noise with different variances:
lik = gpflow.likelihoods.SwitchedLikelihood([gpflow.likelihoods.StudentT(), gpflow.likelihoods.StudentT(), gpflow.likelihoods.StudentT()])
m = gpflow.models.VGP(X_aug, Y_aug, kern=kern, likelihood=lik)
## optimization errors out
gpflow.train.ScipyOptimizer().minimize(m)
 


Discussion:
@awayt039 thanks for spotting this and providing a straightforward example that reproduces the bug (I've edited your question to change `X_aug` and `Y_aug` with random numbers, which gives the same issue, and makes the code easier to look at!) I've not yet figured out where the issue is exactly, though I've got an idea, will update if/when it's clear what's going wrong and how to fix it!

Resolving patch:
diff --git a/gpflow/models/model.py b/gpflow/models/model.py
--- a/gpflow/models/model.py
+++ b/gpflow/models/model.py
@@ -22,6 +22,7 @@
 from ..params import Parameterized, DataHolder
 from ..decors import autoflow
 from ..mean_functions import Zero
+from ..likelihoods import SwitchedLikelihood
 
 
 class Model(Parameterized):
@@ -135,7 +136,22 @@ class GPModel(Model):
     def __init__(self, X, Y, kern, likelihood, mean_function,
                  num_latent=None, name=None):
         super(GPModel, self).__init__(name=name)
-        self.num_latent = num_latent or Y.shape[1]
+        if num_latent is None:
+            # Note: It's not nice for `Model` to need to be aware of specific
+            # likelihoods as here. However, `num_latent` is a bit more broken
+            # in general, specifically regarding multioutput kernels. We
+            # should fix this in the future.
+            # It also has slightly problematic assumptions re the output
+            # dimensions of mean_function.
+            num_latent = Y.shape[1]
+            if isinstance(likelihood, SwitchedLikelihood):
+                # the SwitchedLikelihood partitions/stitches based on the last
+                # column in Y, but we should not add a separate latent GP for
+                # this! hence decrement by 1
+                assert num_latent >= 2
+                num_latent -= 1
+
+        self.num_latent = num_latent
         self.mean_function = mean_function or Zero(output_dim=self.num_latent)
         self.kern = kern
         self.likelihood = likelihood


Test patch:
diff --git a/tests/test_likelihoods.py b/tests/test_likelihoods.py
--- a/tests/test_likelihoods.py
+++ b/tests/test_likelihoods.py
@@ -578,6 +578,21 @@ def test_bad_num_latent(self):
             with self.assertRaises(tf.errors.InvalidArgumentError):
                 m.compute_log_likelihood()  # should die
 
+def test_SwitchedLikelihood_withVGP(session_tf):
+    """
+    Reproduces the bug in https://github.com/GPflow/GPflow/issues/951
+    """
+    X = np.random.randn(12+15, 1)
+    Y = np.random.randn(12+15, 1)
+    idx = np.array([0]*12 + [1]*15)
+    Y_aug = np.c_[Y, idx]
+    assert Y_aug.shape == (12+15, 2)
+
+    kern = gpflow.kernels.Matern32(input_dim=1)
+    lik = gpflow.likelihoods.SwitchedLikelihood([gpflow.likelihoods.StudentT(), gpflow.likelihoods.StudentT()])
+    m = gpflow.models.VGP(X, Y_aug, kern=kern, likelihood=lik)
+    ## optimization errors out
+    gpflow.train.ScipyOptimizer().minimize(m, maxiter=1)
 
 if __name__ == "__main__":
     tf.test.main()