Repository: cornellius-gp/gpytorch
Issue: [Bug] Invalid index in gather during call to mll in training loop

# 🐛 Bug

I followed the docs related to DKL exactly and got the following error during the training loop when calculating the loss with the marginal likelihood function

** Stack trace/error message **
```
Traceback (most recent call last):
  File "gp.py", line 24, in <module>
    exact.train()
  File "/st2/jeff/real_estate/models/gaussian_processes/exact.py", line 102, in train
    loss = -mll(output, train_y).sum()
  File "/st2/jeff/anaconda3/envs/jeff/lib/python3.7/site-packages/gpytorch/module.py", line 22, in __call__
    outputs = self.forward(*inputs, **kwargs)
  File "/st2/jeff/anaconda3/envs/jeff/lib/python3.7/site-packages/gpytorch/mlls/exact_marginal_log_likelihood.py", line 27, in forward
    res = output.log_prob(target)
  File "/st2/jeff/anaconda3/envs/jeff/lib/python3.7/site-packages/gpytorch/distributions/multivariate_normal.py", line 128, in log_prob
    inv_quad, logdet = covar.inv_quad_logdet(inv_quad_rhs=diff.unsqueeze(-1), logdet=True)
  File "/st2/jeff/anaconda3/envs/jeff/lib/python3.7/site-packages/gpytorch/lazy/batch_repeat_lazy_tensor.py", line 242, in inv_quad_logdet
    inv_quad_rhs, logdet, reduce_inv_quad=False
  File "/st2/jeff/anaconda3/envs/jeff/lib/python3.7/site-packages/gpytorch/lazy/lazy_tensor.py", line 1052, in inv_quad_logdet
    *args,
  File "/st2/jeff/anaconda3/envs/jeff/lib/python3.7/site-packages/gpytorch/functions/_inv_quad_log_det.py", line 63, in forward
    preconditioner, precond_lt, logdet_correction = lazy_tsr._preconditioner()
  File "/st2/jeff/anaconda3/envs/jeff/lib/python3.7/site-packages/gpytorch/lazy/added_diag_lazy_tensor.py", line 59, in _preconditioner
    self._piv_chol_self = pivoted_cholesky.pivoted_cholesky(self._lazy_tensor, max_iter)
  File "/st2/jeff/anaconda3/envs/jeff/lib/python3.7/site-packages/gpytorch/utils/pivoted_cholesky.py", line 19, in pivoted_cholesky
    matrix_diag = matrix._approx_diag()
  File "/st2/jeff/anaconda3/envs/jeff/lib/python3.7/site-packages/gpytorch/lazy/interpolated_lazy_tensor.py", line 90, in _approx_diag
    left_res = left_interp(self.left_interp_indices, self.left_interp_values, base_diag_root.unsqueeze(-1))
  File "/st2/jeff/anaconda3/envs/jeff/lib/python3.7/site-packages/gpytorch/utils/interpolation.py", line 187, in left_interp
    res = rhs_expanded.gather(-3, interp_indices_expanded).mul(interp_values_expanded)
RuntimeError: Invalid index in gather at /tmp/pip-req-build-58y_cjjl/aten/src/TH/generic/THTensorEvenMoreMath.cpp:472
loss: 57158.71 med: 0.30, minmax: 0.30 0.30 noise: 0.56: : 0it [00:08, ?it/s]
```

## Expected Behavior

I am left unsure of what is causing the error and how to go about fixing it because it is initially successful in iterating and calculating the loss and then it crashes. The sizing of tensors must be correct, but there is must be some numerical instability and I am unsure about where to look for it.

## System information

**Please complete the following information:**
gpytorch version: `0.3.6`
torch version: `1.2.0
Ubuntu 18.04


Changed files:
gpytorch/utils/interpolation.py

diff --git a/gpytorch/utils/interpolation.py b/gpytorch/utils/interpolation.py
index 893424eb..94e810a5 100644
--- a/gpytorch/utils/interpolation.py
+++ b/gpytorch/utils/interpolation.py
@@ -39,7 +39,7 @@ class Interpolation(object):
         res = res + (((-0.5 * U + 2.5).mul(U) - 4).mul(U) + 2) * U_ge_1_le_2
         return res
 
-    def interpolate(self, x_grid: List[torch.Tensor], x_target: torch.Tensor, interp_points=range(-2, 2)):
+    def interpolate(self, x_grid: List[torch.Tensor], x_target: torch.Tensor, interp_points=range(-2, 2), eps=1e-10):
         if torch.is_tensor(x_grid):
             x_grid = convert_legacy_grid(x_grid)
         num_target_points = x_target.size(0)
@@ -99,7 +99,7 @@ class Interpolation(object):
 
         for i in range(num_dim):
             num_grid_points = x_grid[i].size(0)
-            grid_delta = x_grid[i][1] - x_grid[i][0]
+            grid_delta = (x_grid[i][1] - x_grid[i][0]).clamp_min_(eps)
             # left-bounding grid point in index space
             lower_grid_pt_idxs = torch.floor((x_target[:, i] - x_grid[i][0]) / grid_delta)
             # distance from that left-bounding grid point, again in index space