Repository: sktime/sktime

Issue:
[BUG] evaluate fails with Imputer
**Describe the bug**
Calling `evaluate` with a pipeline and imputer fails.  

**To Reproduce**
```python
import pandas as pd
import numpy as np
from sktime.transformations.series.impute import Imputer
from sktime.forecasting.compose import ForecastingPipeline
from sktime.forecasting.naive import NaiveForecaster
from sktime.forecasting.model_evaluation import evaluate
from sktime.forecasting.model_selection import SlidingWindowSplitter

y = pd.Series(np.random.normal(size=100))
X = pd.DataFrame(np.random.normal(size=(100, 2)))
X.iloc[3, 1] = np.nan

cv = SlidingWindowSplitter(fh=[1, 2, 3])

forecaster = ForecastingPipeline([
    ("impute", Imputer()),
    ("forecast", NaiveForecaster())
])

evaluate(forecaster, cv=cv, y=y, X=X)
```

**Expected behavior**
No error. 

**Versions**
<details>
System:
    python: 3.7.8 | packaged by conda-forge | (default, Jul 31 2020, 02:37:09)  [Clang 10.0.1 ]
executable: /Users/mloning/.conda/envs/sktime-dev/bin/python
   machine: Darwin-19.6.0-x86_64-i386-64bit

Python dependencies:
          pip: 21.2.4
   setuptools: 49.6.0.post20210108
      sklearn: 0.24.2
       sktime: 0.8.1
  statsmodels: 0.12.1
        numpy: 1.19.3
        scipy: 1.6.0
       Cython: 0.29.21
       pandas: 1.2.3
   matplotlib: 3.3.4
       joblib: 1.0.1
        numba: 0.53.1
     pmdarima: 1.8.2
      tsfresh: 0.17.0
</details>

<!-- Thanks for contributing! -->



Discussion:
I get `ValueError: `fh` must not be empty`. From first look:  Seems its coming from the `PolynomialTrendForecaster` which is used in `Imputer` with `method=drift`. I guess it needs `fh` in `fit()`? So we might have to pass `fh` also in `evaluate`?
Yes, I get the same error. 

I think `evaluate` works fine. On a side note: we pass the `fh` twice both to `fit` and `predict` which is unnecessary, passing it only in `fit` should be fine. 

I'm not sure, but think the issue is in `_impute_with_forecaster` where we construct the `fh` from the index of missing values (line 207), can it not be that for some columns there are no missing values and hence we get an empty `fh`?
ah yep, we should then probably first check if there is anything to impute at all, if not we even dont need to use the `PolynomialTrendForecaster`. Also it would be faster for `Imputer`. Good catch, thx @mloning :) 
I guess the issue might also be related to the case that you are using multivariate here. I will try to fix it in next days if I have time :)

Resolving patch:
diff --git a/sktime/transformations/series/impute.py b/sktime/transformations/series/impute.py
--- a/sktime/transformations/series/impute.py
+++ b/sktime/transformations/series/impute.py
@@ -9,14 +9,13 @@
 
 import numpy as np
 import pandas as pd
-
 from sklearn.base import clone
 from sklearn.utils import check_random_state
 
+from sktime.forecasting.base import ForecastingHorizon
+from sktime.forecasting.trend import PolynomialTrendForecaster
 from sktime.transformations.base import _SeriesToSeriesTransformer
 from sktime.utils.validation.series import check_series
-from sktime.forecasting.trend import PolynomialTrendForecaster
-from sktime.forecasting.base import ForecastingHorizon
 
 
 class Imputer(_SeriesToSeriesTransformer):
@@ -111,7 +110,7 @@ def transform(self, Z, X=None):
         if not _has_missing_values(Z):
             return Z
 
-        elif self.method == "random":
+        if self.method == "random":
             if isinstance(Z, pd.DataFrame):
                 for col in Z:
                     Z[col] = Z[col].apply(
@@ -203,15 +202,16 @@ def _impute_with_forecaster(forecaster, Z):
         series = [Z[column] for column in Z]
 
     for z in series:
-        # define fh based on index of missing values
-        na_index = z.index[z.isna()]
-        fh = ForecastingHorizon(values=na_index, is_relative=False)
+        if _has_missing_values(z):
+            # define fh based on index of missing values
+            na_index = z.index[z.isna()]
+            fh = ForecastingHorizon(values=na_index, is_relative=False)
 
-        # fill NaN before fitting with ffill and backfill (heuristic)
-        forecaster.fit(y=z.fillna(method="ffill").fillna(method="backfill"), fh=fh)
+            # fill NaN before fitting with ffill and backfill (heuristic)
+            forecaster.fit(y=z.fillna(method="ffill").fillna(method="backfill"), fh=fh)
 
-        # replace missing values with predicted values
-        z[na_index] = forecaster.predict()
+            # replace missing values with predicted values
+            z[na_index] = forecaster.predict()
     return Z
 
 


Test patch:
diff --git a/sktime/transformations/series/tests/test_imputer.py b/sktime/transformations/series/tests/test_imputer.py
new file mode 100644
--- /dev/null
+++ b/sktime/transformations/series/tests/test_imputer.py
@@ -0,0 +1,50 @@
+#!/usr/bin/env python3 -u
+# -*- coding: utf-8 -*-
+# copyright: sktime developers, BSD-3-Clause License (see LICENSE file)
+"""Unit tests of Imputer functionality."""
+
+__author__ = ["aiwalter"]
+__all__ = []
+
+import numpy as np
+import pytest
+
+from sktime.forecasting.exp_smoothing import ExponentialSmoothing
+from sktime.transformations.series.impute import Imputer
+from sktime.utils._testing.forecasting import make_forecasting_problem
+
+y, X = make_forecasting_problem(make_X=True)
+
+X.iloc[3, 0] = np.nan
+X.iloc[3, 1] = np.nan
+X.iloc[0, 1] = np.nan
+X.iloc[-1, 1] = np.nan
+
+y.iloc[3] = np.nan
+y.iloc[0] = np.nan
+y.iloc[-1] = np.nan
+
+
+@pytest.mark.parametrize("Z", [y, X])
+@pytest.mark.parametrize(
+    "method",
+    [
+        "drift",
+        "linear",
+        "nearest",
+        "constant",
+        "mean",
+        "median",
+        "backfill",
+        "pad",
+        "random",
+        "forecaster",
+    ],
+)
+def test_imputer(method, Z):
+    """Test univariate and multivariate Imputer with all methods."""
+    forecaster = ExponentialSmoothing() if method == "forecaster" else None
+    value = 3 if method == "constant" else None
+    t = Imputer(method=method, forecaster=forecaster, value=value)
+    y_hat = t.fit_transform(Z)
+    assert not y_hat.isnull().to_numpy().any()