Repository: sktime/sktime

Issue:
[BUG] evaluate raises KeyError for AutoETS and CutoffSplitter / SlidingWindowSplitter - issue with BaseForecaster._update_y_X
**Describe the bug**
evaluate function raises an invalid error for the CutoffSplitter and SlidingWindowSplitter. 
ExpandingWindowSplitter works fine.

The error initially appears to be in the _predict method of ets.
But, a few debug prints later I managed to pinpoint the issue in the [_update_y_X](https://github.com/alan-turing-institute/sktime/blob/922cd71d0d82d849025a080be826cf1c3c4777e5/sktime/forecasting/base/_base.py#L804) method of the BaseForecaster class.

If the forecaster was fitted before and is now refitted, forecaster._y is not changed based on the new y_train and this causes a difference in the absolute horizon of y and the prediction output. This explains why we don't get the error in the ExpandingWindowSplitter as in that case forecaster._y[0] is the same for all splits.

**To Reproduce**
```python
import numpy as np
import pandas as pd
from sktime.forecasting.ets import AutoETS
from sktime.forecasting.model_selection import CutoffSplitter
from sktime.forecasting.model_evaluation import evaluate
from sktime.performance_metrics.forecasting import MeanSquaredScaledError

synthetic_data = pd.Series(
    10 + np.random.lognormal(5, 1, 365),
    index = pd.PeriodIndex(pd.date_range(start="2020-01-01", freq="D", periods=365), freq="D"), 
)

custom_cutoffs = np.array([
    len(synthetic_data[synthetic_data.index < date]) 
    for date in 
    [
        "2020-07-01",
        "2020-08-01", 
        "2020-09-01", 
        "2020-10-01", 
        "2020-11-01", 
        "2020-12-01",
    ]
])

# create the CV object
CV = CutoffSplitter(
    # custom cuttoffs
    cutoffs=custom_cutoffs, 
    # 3 step ahead forecast
    fh=np.arange(3)+1,
    # 5 years of training data
    window_length=6*30,
)

evaluate(
    forecaster=AutoETS(),
    y=synthetic_data,
    cv=CV,
    strategy="refit",
    return_data=True,
    scoring=MeanSquaredScaledError(square_root=True),
)

>>>  KeyError: "None of [PeriodIndex(['2020-08-02', '2020-08-03', '2020-08-04'], dtype='period[D]')] are in the [index]"
```

**Expected behavior**
evaluate should run smoothly for all cv strategies.
BaseForecaster._y should be updated based on the latest training data. 
I suggest we remove the combine_first logic from  [_update_y_X](https://github.com/alan-turing-institute/sktime/blob/922cd71d0d82d849025a080be826cf1c3c4777e5/sktime/forecasting/base/_base.py#L804), but I'm not sure if it serves a different purpose elsewhere.

Otherwise a deepcopy(forecaster) in evaluate will solve the problem too.


Discussion:

Resolving patch:
diff --git a/sktime/forecasting/base/_base.py b/sktime/forecasting/base/_base.py
--- a/sktime/forecasting/base/_base.py
+++ b/sktime/forecasting/base/_base.py
@@ -803,7 +803,7 @@ def _update_y_X(self, y, X=None, enforce_index_type=None):
         # we only need to modify _y if y is not None
         if y is not None:
             # if _y does not exist yet, initialize it with y
-            if not hasattr(self, "_y") or self._y is None:
+            if not hasattr(self, "_y") or self._y is None or not self.is_fitted:
                 self._y = y
             # otherwise, update _y with the new rows in y
             #  if y is np.ndarray, we assume all rows are new
@@ -819,7 +819,7 @@ def _update_y_X(self, y, X=None, enforce_index_type=None):
         # we only need to modify _X if X is not None
         if X is not None:
             # if _X does not exist yet, initialize it with X
-            if not hasattr(self, "_X") or self._X is None:
+            if not hasattr(self, "_X") or self._X is None or not self.is_fitted:
                 self._X = X
             # otherwise, update _X with the new rows in X
             #  if X is np.ndarray, we assume all rows are new


Test patch:
diff --git a/sktime/forecasting/tests/test_all_forecasters.py b/sktime/forecasting/tests/test_all_forecasters.py
--- a/sktime/forecasting/tests/test_all_forecasters.py
+++ b/sktime/forecasting/tests/test_all_forecasters.py
@@ -416,3 +416,18 @@ def test_update_predict_predicted_index_update_params(
     _check_update_predict_predicted_index(
         Forecaster, fh, window_length, step_length, update_params
     )
+
+
+# test that _y is updated when forecaster is refitted
+@pytest.mark.parametrize("Forecaster", FORECASTERS)
+def test__y_when_refitting(Forecaster):
+    f = _construct_instance(Forecaster)
+    columns = _get_n_columns(f.get_tag("scitype:y"))
+    for n_columns in columns:
+        f = _construct_instance(Forecaster)
+        y_train = _make_series(n_columns=n_columns)
+        f.fit(y_train, fh=FH0)
+        f.fit(y_train[3:], fh=FH0)
+        # using np.squeeze to make the test flexible to shape differeces like
+        # (50,) and (50, 1)
+        assert np.all(np.squeeze(f._y) == np.squeeze(y_train[3:]))