Repository: sktime/sktime

Issue:
[BUG] Naive model with last strategy is sensitive to trailing NaN values
**Describe the bug**
<!--
A clear and concise description of what the bug is.
-->
The naive model with "last" strategy really picks the last value, even if that's a NaN value. Can we make this more robust, similar to how the "mean" strategy actually does a `nanmean` under the hood? I could offer a PR to make it pick the last non-NaN value, taking into account the requested seasonal periodicity.

**To Reproduce**
<!--
Add a Minimal, Complete, and Verifiable example (for more details, see e.g. https://stackoverflow.com/help/mcve

If the code is too long, feel free to put it in a public gist and link it in the issue: https://gist.github.com
-->

```python
import numpy as np
import pandas as pd
from sktime.forecasting.base import ForecastingHorizon
from sktime.forecasting.naive import NaiveForecaster

data = pd.Series(
    index=pd.date_range(
        "2021-06-01 00:00", periods=12, freq="1h", closed="left"
    ),
    data=[1, 2, 3, 1, 2, 3, np.nan, 2, 3, 1, 2, 3]
)
train_data = data[:7]
test_data = data[7:]

print(train_data)
# 2021-06-01 00:00:00    1.0
# 2021-06-01 01:00:00    2.0
# 2021-06-01 02:00:00    3.0
# 2021-06-01 03:00:00    1.0
# 2021-06-01 04:00:00    2.0
# 2021-06-01 05:00:00    3.0
# 2021-06-01 06:00:00    NaN
# Freq: H, dtype: float64

fh = ForecastingHorizon(test_data.index, is_relative=False)
model = NaiveForecaster(strategy="last", sp=3)
model.fit(train_data)
forecast_data = model.predict(fh)

print(forecast_data)
# 2021-06-01 07:00:00    2.0
# 2021-06-01 08:00:00    3.0
# 2021-06-01 09:00:00    NaN  # Let's make this a 1
# 2021-06-01 10:00:00    2.0
# 2021-06-01 11:00:00    3.0
# dtype: float64
```

**Expected behavior**
<!--
A clear and concise description of what you expected to happen.
-->

**Additional context**
<!--
Add any other context about the problem here.
-->

**Versions**
<details>

<!--
Please run the following code snippet and paste the output here:
 
from sktime import show_versions; show_versions()
-->

</details>

<!-- Thanks for contributing! -->



Discussion:
Hi @Flix6x, yes that makes sense, would appreciate the PR! Let me know if you need help with the recently introduced docstring tests which will require some fixes beyond your suggested changes I believe. 

We should also think about adding that to the input checks (e.g. `check_y(..., allow_nan=True)` which by default is set `False` and raises an error if data with missing data is passed, but that may best be addressed in a separate PR. 


Resolving patch:
diff --git a/sktime/forecasting/naive.py b/sktime/forecasting/naive.py
--- a/sktime/forecasting/naive.py
+++ b/sktime/forecasting/naive.py
@@ -4,7 +4,7 @@
 """Implements simple forecasts based on naive assumptions."""
 
 __all__ = ["NaiveForecaster"]
-__author__ = ["Markus Löning", "Piyush Gade"]
+__author__ = ["mloning", "Piyush Gade", "Flix6x"]
 
 from warnings import warn
 
@@ -20,27 +20,40 @@ class NaiveForecaster(_BaseWindowForecaster):
     """Forecast based on naive assumptions about past trends continuing.
 
     NaiveForecaster is a forecaster that makes forecasts using simple
-    strategies.
+    strategies. Two out of three strategies are robust against NaNs.
+
+    Internally, this forecaster does the following:
+    - obtains the so-called "last window", a 1D array that denotes the
+      most recent time window that the forecaster is allowed to use
+    - reshapes the last window into a 2D array according to the given
+      seasonal periodicity (prepended with NaN values to make it fit);
+    - make a prediction for each column, using the given strategy:
+      - "last": last non-NaN row
+      - "mean": np.nanmean over rows
+    - tile the predictions using the seasonal periodicity
 
     Parameters
     ----------
     strategy : {"last", "mean", "drift"}, default="last"
         Strategy used to make forecasts:
 
-        * "last" : forecast the last value in the
+        * "last":   (robust against NaN values)
+                    forecast the last value in the
                     training series when sp is 1.
                     When sp is not 1,
                     last value of each season
                     in the last window will be
                     forecasted for each season.
-        * "mean" : forecast the mean of last window
-                     of training series when sp is 1.
-                     When sp is not 1, mean of all values
-                     in a season from last window will be
-                     forecasted for each season.
-        * "drift": forecast by fitting a line between the
+        * "mean":   (robust against NaN values)
+                    forecast the mean of last window
+                    of training series when sp is 1.
+                    When sp is not 1, mean of all values
+                    in a season from last window will be
+                    forecasted for each season.
+        * "drift":  (not robust against NaN values)
+                    forecast by fitting a line between the
                     first and last point of the window and
-                     extrapolating it into the future.
+                    extrapolating it into the future.
 
     sp : int, default=1
         Seasonal periodicity to use in the seasonal forecasting.
@@ -60,7 +73,10 @@ class NaiveForecaster(_BaseWindowForecaster):
     >>> y_pred = forecaster.predict(fh=[1,2,3])
     """
 
-    _tags = {"requires-fh-in-fit": False}
+    _tags = {
+        "requires-fh-in-fit": False,
+        "handles-missing-data": False,  # todo: switch to True if GH1367 is fixed
+    }
 
     def __init__(self, strategy="last", window_length=None, sp=1):
         super(NaiveForecaster, self).__init__()
@@ -68,6 +84,11 @@ def __init__(self, strategy="last", window_length=None, sp=1):
         self.sp = sp
         self.window_length = window_length
 
+        # Override tag for handling missing data
+        # todo: remove if GH1367 is fixed
+        if self.strategy in ("last", "mean"):
+            self.set_tags(**{"handles-missing-data": True})
+
     def _fit(self, y, X=None, fh=None):
         """Fit to training data.
 
@@ -88,25 +109,8 @@ def _fit(self, y, X=None, fh=None):
 
         n_timepoints = y.shape[0]
 
-        if self.strategy == "last":
-            if self.sp == 1:
-                if self.window_length is not None:
-                    warn(
-                        "For the `last` strategy, "
-                        "the `window_length` value will be ignored if `sp` "
-                        "== 1."
-                    )
-                self.window_length_ = 1
-
-            else:
-                self.sp_ = check_sp(self.sp)
-
-                # window length we need for forecasts is just the
-                # length of seasonal periodicity
-                self.window_length_ = self.sp_
-
-        elif self.strategy == "mean":
-            # check window length is greater than sp for seasonal mean
+        if self.strategy in ("last", "mean"):
+            # check window length is greater than sp for seasonal mean or seasonal last
             if self.window_length is not None and self.sp != 1:
                 if self.window_length < self.sp:
                     raise ValueError(
@@ -117,7 +121,7 @@ def _fit(self, y, X=None, fh=None):
             self.window_length_ = check_window_length(self.window_length, n_timepoints)
             self.sp_ = check_sp(self.sp)
 
-            #  if not given, set default window length for the mean strategy
+            #  if not given, set default window length
             if self.window_length is None:
                 self.window_length_ = len(y)
 
@@ -168,56 +172,41 @@ def _predict_last_window(
 
         elif self.strategy == "last":
             if self.sp == 1:
-                return np.repeat(last_window[-1], len(fh))
+                last_valid_value = last_window[
+                    (~np.isnan(last_window))[0 :: self.sp].cumsum().argmax()
+                ]
+                return np.repeat(last_valid_value, len(fh))
 
             else:
-                # we need to replicate the last window if max(fh) is larger
-                # than sp,so that we still make forecasts by repeating the
-                # last value for that season, assume fh is sorted, i.e. max(
-                # fh) == fh[-1]
-                if fh[-1] > self.sp_:
-                    reps = np.int(np.ceil(fh[-1] / self.sp_))
-                    last_window = np.tile(last_window, reps=reps)
-
-                # get zero-based index by subtracting the minimum
-                fh_idx = fh.to_indexer(self.cutoff)
-                return last_window[fh_idx]
+                # reshape last window, one column per season
+                last_window = self._reshape_last_window_for_sp(last_window)
+
+                # select last non-NaN row for every column
+                y_pred = last_window[
+                    (~np.isnan(last_window)).cumsum(0).argmax(0).T,
+                    range(last_window.shape[1]),
+                ]
+
+                # tile prediction according to seasonal periodicity
+                y_pred = self._tile_seasonal_prediction(y_pred, fh)
+
+                return y_pred
 
         elif self.strategy == "mean":
             if self.sp == 1:
                 return np.repeat(np.nanmean(last_window), len(fh))
 
             else:
-                # if the window length is not a multiple of sp, we pad the
-                # window with nan values for easy computation of the mean
-                remainder = self.window_length_ % self.sp_
-                if remainder > 0:
-                    pad_width = self.sp_ - remainder
-                else:
-                    pad_width = 0
-                last_window = np.hstack([np.full(pad_width, np.nan), last_window])
-
                 # reshape last window, one column per season
-                last_window = last_window.reshape(
-                    np.int(np.ceil(self.window_length_ / self.sp_)), self.sp_
-                )
+                last_window = self._reshape_last_window_for_sp(last_window)
 
-                # compute seasonal mean, averaging over rows
+                # compute seasonal mean, averaging over non-NaN rows for every column
                 y_pred = np.nanmean(last_window, axis=0)
 
-                # we need to replicate the last window if max(fh) is
-                # larger than sp,
-                # so that we still make forecasts by repeating the
-                # last value for that season,
-                # assume fh is sorted, i.e. max(fh) == fh[-1]
-                # only slicing all the last seasons into last_window
-                if fh[-1] > self.sp_:
-                    reps = np.int(np.ceil(fh[-1] / self.sp_))
-                    y_pred = np.tile(y_pred, reps=reps)
+                # tile prediction according to seasonal periodicity
+                y_pred = self._tile_seasonal_prediction(y_pred, fh)
 
-                # get zero-based index by subtracting the minimum
-                fh_idx = fh.to_indexer(self.cutoff)
-                return y_pred[fh_idx]
+                return y_pred
 
         # if self.strategy == "drift":
         else:
@@ -240,3 +229,55 @@ def _predict_last_window(
                     # linear extrapolation
                     y_pred = last_window[-1] + (fh_idx + 1) * slope
                     return y_pred
+
+    def _reshape_last_window_for_sp(self, last_window):
+        """Reshape the 1D last window into a 2D last window, prepended with NaN values.
+
+        The 2D array has 1 column per season.
+
+        For example:
+
+            last_window = [1, 2, 3, 4]
+            sp = 3  # i.e. 3 distinct seasons
+            reshaped_last_window = [[nan, nan, 1],
+                                    [  2,   3, 4]]
+        """
+        # if window length is not multiple of sp, backward fill window with nan values
+        remainder = self.window_length_ % self.sp_
+        if remainder > 0:
+            pad_width = self.sp_ - remainder
+        else:
+            pad_width = 0
+        last_window = np.hstack([np.full(pad_width, np.nan), last_window])
+
+        # reshape last window, one column per season
+        last_window = last_window.reshape(
+            np.int(np.ceil(self.window_length_ / self.sp_)), self.sp_
+        )
+
+        return last_window
+
+    def _tile_seasonal_prediction(self, y_pred, fh):
+        """Tile a prediction to cover all requested forecasting horizons.
+
+        The original prediction has 1 value per season.
+
+        For example:
+
+            fh = [1, 2, 3, 4, 5, 6, 7]
+            y_pred = [2, 3, 1]  # note len(y_pred) = sp
+            y_pred_tiled = [2, 3, 1, 2, 3, 1, 2]
+        """
+        # we need to replicate the last window if max(fh) is
+        # larger than sp,
+        # so that we still make forecasts by repeating the
+        # last value for that season,
+        # assume fh is sorted, i.e. max(fh) == fh[-1]
+        # only slicing all the last seasons into last_window
+        if fh[-1] > self.sp_:
+            reps = np.int(np.ceil(fh[-1] / self.sp_))
+            y_pred = np.tile(y_pred, reps=reps)
+
+        # get zero-based index by subtracting the minimum
+        fh_idx = fh.to_indexer(self.cutoff)
+        return y_pred[fh_idx]


Test patch:
diff --git a/sktime/forecasting/tests/test_naive.py b/sktime/forecasting/tests/test_naive.py
--- a/sktime/forecasting/tests/test_naive.py
+++ b/sktime/forecasting/tests/test_naive.py
@@ -1,8 +1,9 @@
 #!/usr/bin/env python3 -u
 # -*- coding: utf-8 -*-
 # copyright: sktime developers, BSD-3-Clause License (see LICENSE file)
+"""Tests simple forecasts based on naive assumptions."""
 
-__author__ = ["Markus Löning", "Piyush Gade"]
+__author__ = ["mloning", "Piyush Gade", "Flix6x"]
 
 import numpy as np
 import pandas as pd
@@ -25,6 +26,7 @@
 
 @pytest.mark.parametrize("fh", TEST_OOS_FHS)
 def test_strategy_last(fh):
+    """Test last strategy."""
     f = NaiveForecaster(strategy="last")
     f.fit(y_train)
     y_pred = f.predict(fh)
@@ -35,6 +37,7 @@ def test_strategy_last(fh):
 @pytest.mark.parametrize("fh", TEST_OOS_FHS)
 @pytest.mark.parametrize("window_length", TEST_WINDOW_LENGTHS)
 def test_strategy_mean(fh, window_length):
+    """Test mean strategy."""
     f = NaiveForecaster(strategy="mean", window_length=window_length)
     f.fit(y_train)
     y_pred = f.predict(fh)
@@ -49,6 +52,7 @@ def test_strategy_mean(fh, window_length):
 @pytest.mark.parametrize("fh", TEST_OOS_FHS)
 @pytest.mark.parametrize("sp", TEST_SPS)
 def test_strategy_last_seasonal(fh, sp):
+    """Test last strategy on seasonal data."""
     f = NaiveForecaster(strategy="last", sp=sp)
     f.fit(y_train)
     y_pred = f.predict(fh)
@@ -67,6 +71,7 @@ def test_strategy_last_seasonal(fh, sp):
 @pytest.mark.parametrize("sp", TEST_SPS)
 @pytest.mark.parametrize("window_length", [*TEST_WINDOW_LENGTHS, None])
 def test_strategy_mean_seasonal(fh, sp, window_length):
+    """Test mean strategy on seasonal data."""
     if (window_length is not None and window_length > sp) or (window_length is None):
         f = NaiveForecaster(strategy="mean", sp=sp, window_length=window_length)
         f.fit(y_train)
@@ -97,8 +102,7 @@ def test_strategy_mean_seasonal(fh, sp, window_length):
 @pytest.mark.parametrize("n_seasons", [1, 3])
 @pytest.mark.parametrize("sp", TEST_SPS)
 def test_strategy_mean_seasonal_simple(n_seasons, sp):
-    # create 2d matrix, rows are different seasons, columns time points of
-    # each season
+    """Create 2d matrix (seasons on rows, time points of each season on columns)."""
     values = np.random.normal(size=(n_seasons, sp))
     y = pd.Series(values.ravel())
 
@@ -116,7 +120,7 @@ def test_strategy_mean_seasonal_simple(n_seasons, sp):
 @pytest.mark.parametrize("fh", TEST_OOS_FHS)
 @pytest.mark.parametrize("window_length", [*TEST_WINDOW_LENGTHS, None])
 def test_strategy_drift_unit_slope(fh, window_length):
-    # drift strategy for constant slope 1
+    """Test drift strategy for constant slope 1."""
     if window_length != 1:
         f = NaiveForecaster(strategy="drift", window_length=window_length)
         f.fit(y_train)
@@ -135,7 +139,7 @@ def test_strategy_drift_unit_slope(fh, window_length):
 @pytest.mark.parametrize("fh", TEST_OOS_FHS)
 @pytest.mark.parametrize("window_length", [*TEST_WINDOW_LENGTHS, None])
 def test_strategy_drift_flat_line(fh, window_length):
-    # test for flat time series data
+    """Test flat time series data."""
     if window_length != 1:
         y_train = pd.Series(np.ones(20))
         f = NaiveForecaster(strategy="drift", window_length=window_length)
@@ -155,7 +159,7 @@ def test_strategy_drift_flat_line(fh, window_length):
 @pytest.mark.parametrize("fh", TEST_OOS_FHS)
 @pytest.mark.parametrize("window_length", [*TEST_WINDOW_LENGTHS, None])
 def test_strategy_drift_window_length(fh, window_length):
-    # test for checking if window_length is properly working
+    """Test whether window_length is properly working."""
     if window_length != 1:
         if window_length is None:
             window_length = len(y_train)
@@ -178,12 +182,19 @@ def test_strategy_drift_window_length(fh, window_length):
 @pytest.mark.parametrize("n", [3, 5])
 @pytest.mark.parametrize("window_length", list({4, 5, *TEST_WINDOW_LENGTHS}))
 @pytest.mark.parametrize("sp", list({3, 4, 8, *TEST_SPS}))
-def test_strategy_mean_seasonal_additional_combinations(n, window_length, sp):
-    """Check time series of n * window_length with a 1:n-1 train/test split,
+@pytest.mark.parametrize("strategy", ["last", "mean"])
+def test_strategy_mean_and_last_seasonal_additional_combinations(
+    n, window_length, sp, strategy
+):
+    """Check that naive forecasters yield the right forecasts given simple data.
+
+    Test for perfectly cyclic data, and for robustness against a missing value.
+    More specifically,
+    check time series of n * window_length with a 1:n-1 train/test split,
     for different combinations of the period and seasonal periodicity.
-    The time series contains perfectly cyclic data.
+    The time series contains perfectly cyclic data,
+    so switching between the "mean" and "last" strategies should not make a difference.
     """
-
     # given <window_length> hours of data with a seasonal periodicity of <sp> hours
     freq = pd.Timedelta("1H")
     data = pd.Series(
@@ -195,6 +206,11 @@ def test_strategy_mean_seasonal_additional_combinations(n, window_length, sp):
         ],
     )
 
+    # For selected cases, remove a redundant data point by making it NaN
+    if window_length > sp:
+        # create a trailing NaN value in the training set
+        data[window_length - 1] = np.nan
+
     # Split into train and test data
     train_data = data[:window_length]
     test_data = data[window_length:]
@@ -202,28 +218,31 @@ def test_strategy_mean_seasonal_additional_combinations(n, window_length, sp):
     # Forecast data does not retain the original frequency
     test_data.index.freq = None
 
-    # For example, for n=2, periods=4 and sp=3:
+    # For example, for n=2, window_length=4, sp=3:
 
     # print(train_data)
     # 2021-06-01 00:00:00    1.0
     # 2021-06-01 01:00:00    2.0
     # 2021-06-01 02:00:00    3.0
-    # 2021-06-01 03:00:00    1.0
+    # 2021-06-01 03:00:00    NaN
     # Freq: H, dtype: int64
 
     # print(test_data)
     # 2021-06-01 04:00:00    2.0  # (value of 3 hours earlier)
     # 2021-06-01 05:00:00    3.0  # (value of 3 hours earlier)
-    # 2021-06-01 06:00:00    1.0  # (mean value of 3 and 6 hours earlier)
+    # 2021-06-01 06:00:00    1.0  # (value of 6 hours earlier)
     # 2021-06-01 07:00:00    2.0  # (value of 6 hours earlier)
     # dtype: float64
 
-    # let's forecast the next <2 x period> hours with a periodicity of <sp> hours
+    # forecast the next <(n-1) x window_length> hours with periodicity of <sp> hours
     fh = ForecastingHorizon(test_data.index, is_relative=False)
-    model = NaiveForecaster(strategy="mean", sp=sp)
+    model = NaiveForecaster(strategy=strategy, sp=sp)
     model.fit(train_data)
     forecast_data = model.predict(fh)
 
+    # Make sure that the model (object) reports that it handles missing data
+    assert model.get_tag("handles-missing-data")
+
     if sp < window_length:
         # We expect a perfect forecast given our perfectly cyclic data
         pd.testing.assert_series_equal(forecast_data, test_data)