Repository: sktime/sktime

Issue:
[BUG] bug in `NaiveForecaster` variance formulae?
From https://github.com/alan-turing-institute/sktime/pull/3435/files which I've merged and only now noticed potential issues in the formula for `predict_var`.

The main issue (which I missed on first read) was that I think we need rolling forecasts, no?

* should the `mean` strategy not have rolling mean predictions used, instead of the single prediction at the `cutoff`? So, `y.rolling(window_length).mean` instead of y_pred? With an appropriate shift per `fh`?
* `last` seems correct only for `fh` indices that are smaller or equal to `sp`.
* the `slope` strategy should use `window_length` instead of `T`, and also do this on rolling basis?

FYI @topher-lo


Discussion:
So, for instance, for `"last"`:

```python
if self.strategy == "last":
    for h in fh:  # this may need conversion to integer index first
        sp = self.sp
        numsp = ceil(h / sp)
        y_res[h] = y - y.shift(numsp * sp)
```
then you use `y_res[h]` to compute the predictive variance at horizon `h`
@fkiraly An oversight on our part as my team haven't used `window_length` in our existing codebase and I don't think it's an argument in the R implementation either. We will sort this issue out **before the end of the day.** Seems relatively straightforward imo.

1. `mean`: Use pandas ~rolling~ expanding mean
2. `last`:  Please see @jasmineliaw's comment below
3. `drift`: We can make it rolling 👍

One last point, I think `drift` rolling operations can be expressed purely in matrix notation. We will implement this in a fully vectorized way without the need for for loops.
For `last` method with fh > sp, we compared the result from our implementation with R and both produced the same result.

Refer to screenshots below for results from R and our implementation respectively, tested using sp=24 and fh=30.

- From R:
![image](https://user-images.githubusercontent.com/93530957/191888350-b30dfeb4-7369-4bb9-9f26-37f5a0727230.png)

- From sktime:
![image](https://user-images.githubusercontent.com/93530957/191889415-6892fbc9-fea6-42d7-b46c-628d291aad23.png)

To reproduce, please refer to below:

- From R:
```R
library(fpp3)
library(tsibbledata)

# export y to csv from sktime example below
data <- read.csv("y.csv")
data$X <- seq(as.POSIXct("1980-01-01 00:00:00"), as.POSIXct("1991-05-29 16:00:00"), by="hour")
data <- as_tsibble(data, index=X)

y_pred <- data %>% model(SNAIVE(X0 ~ lag(24))) %>%
  forecast(h=30) %>%
  hilo(90)
```

- From sktime:
```python
n_timepoints = 100000
mu, sigma = 0.0, 10.0
fake_idx = pd.date_range("1980", periods=n_timepoints + 1, freq="H")
np.random.seed(42)
y = pd.Series(np.random.normal(mu, sigma, size=n_timepoints), index=fake_idx[:-1])

fh = list(range(1, 31))
window_length = None
coverage = 0.90
forecaster = NaiveForecaster("last", sp=24, window_length=window_length)

y_pred_ints = forecaster.fit(y).predict_interval(fh=fh, coverage=coverage)
```



> * should the `mean` strategy not have rolling mean predictions used, instead of the single prediction at the `cutoff`? So, `y.rolling(window_length).mean` instead of y_pred? With an appropriate shift per `fh`?

I think the appropriate shift here is just `step_length = 1` for any `fh`. From what I've seen in practice (in productionized finance forecasting pipelines), naive methods are cheap computationally and are almost always retrained for every period.
@jasmineliaw Could you link over code from the R implementation for computing residuals for the `last` (with sp > 1) method? 
> @jasmineliaw Could you link over code from the R implementation for computing residuals for the `last` (with sp > 1) method?

We have referred to R implementation from below:
- https://github.com/robjhyndman/forecast/blob/master/R/naive.R#L27

```R
fitted <- ts(c(rep(NA, lag), head(fits, -lag)), start = start(y), frequency = m)
```

- https://github.com/robjhyndman/forecast/blob/master/R/naive.R#L39
```R
    res <- y - fitted
```

@fkiraly Do let us know if you have any other suggestions for the `last` method.

hm... ok, if it agrees with the R version, I would say, let's fix `window_length` and leave the PR merged in - but continue discussing here until we have satisfied ourselves that the implemented formulae are indeed correct.

Given the agreement with the R version, this will probably mean that either none of the two implementations have buggy math, or both. The latter is acceptable as a temporary state since `forecast` is a "reference" package.

We were planning to advertise this as a highlight, but let's also hold off on that until we have clarified the situation.
I'll continue in deriving the math and then we can make a comparison.
For math, let's first analyze a stationary residuals process, say, the actual/observable time series is $s(t) = a(t) + \epsilon(t)$ for time stamps $t\in T$ (regularly spaced), with stochastic centered noise $\epsilon$ (all quantities are time series valued random variables).

Let's consider an algorithm that produces forecasts at horizon $h$, i.e., we have forecasts $\widehat{s}_h(t)$ for all $t$ that depend
deterministically on all $s(t')$ with $t'\le t-h$. Write $r_h(t) := s(t) - \widehat{s}_h(t)$ for the forecast residuals at horizon $h$.

Note these are all residuals *at* a forecast horizon $h$. Often it is assumed (and let's do that for simplicity) that $r_h(t)$ is variance stationary, so $\mbox{Var } r_h(t)$ does not depend on $t$, but it depends on $h$, so let's call this $\sigma_h^2$.

Let's say we have these residuals at time stamps $t\in R$. The typical estimate for $\sigma_h^2$ would be $\frac{1}{ | R | \cdot \rho_h } \sum_{t \in R} (r_h(t) - \bar{r}_h)^2$, where $\bar{r}_h$ is the sample mean of the $r_h(t)$, ranging over $t\in R$, and $\rho_h$ is effective sample size correction coefficient of $r_h$ (this can be estimated in various ways).

What is important is not how we compute $\rho_h$ (there are different ways), but the sum we need to compute (which does not depend on $\rho_h$).

For the naive forecaster, the sum would be computed from `y - y.shift(h)` for "last value carry forward" (in the non-seasonal case), and we get one $\sigma_h^2$ per $h$ in the `sktime` forecasting horizon vector `fh`. Similar for rolling mean etc.

Which is not what the current code does: it does not loop over $h$-es, and it does not look at the right residuals.
@fkiraly I just started a long writeup on this issue. Would you be available on Discord for a quick call right now?
I think the problem is more fundamental and on the `sktime` side.
> @fkiraly I just started a long writeup on this issue. Would you be available on Discord for a quick call right now?

Not right now (have to get to work), but the forecasting meet-up and the collab session will take place (see schedule in slack general channel), I'll try to make it. If that doesn't work, we can try later on the day.
> I think the problem is more fundamental and on the `sktime` side.

That's ominous, so it would be great if you could spell out what you mean.
@fkiraly I dug deeper into the R implementation and had a deeper look at the theory behind "naive" methods. I take back my [points above](https://github.com/alan-turing-institute/sktime/issues/3469#issuecomment-1255737350). I think there is a more subtle statistical discussion to be had and one which I think `sktime` has made a crucial mistake in for their `NaiveForecaster` implementation.
1. In my mind, a "mean" naive strategy is fundamentally different to a "moving average" forecaster. The former is making a assumption that the time series's data generating process follows from a straight line population "mean" and every deviation from this constant population mean are I.I.D. random shocks. Please correct me if I'm wrong, but it seems that sktime has muddled up this idea with a "K moving average" aka `ARIMA(0, 0, K)` model, which assumes serial correlation. If you look into the R implementation, you'll notice that they do NOT have an option for `window_length` in their naive forecaster implementation. Furthermore, if you look at the chart from the FPP3 textbook on the mean method, it's just a straight line across across the whole time series: https://otexts.com/fpp3/simple-methods.html#mean-method
>  one which I think `sktime` has made a crucial mistake in for their `NaiveForecaster` implementation.

Hm, there are eight authors listed. I only did software refactoring when interfaces changed but should probably have looked at the math, too. Generally, we should spell out in the docstring what the math is especially when we implement simple methods and don't interface them.

Pinging everyone listed as author: @mloning, @Piyush1729, @sri1419, @Flix6x, @aiwalter, @IlyasMoutawwakil, @bethrice44 (I think @IlyasMoutawwakil and @bethrice44 are there for `NaiveVariance`, but doesn't matter since I know they know how to work with math 😄 )


> For math, let's first analyze a stationary residuals process, say, the actual/observable time series is s(t)=a(t)+ϵ(t) for time stamps t∈T (regularly spaced), with stochastic centered noise ϵ (all quantities are time series valued random variables).
> 
> Let's consider an algorithm that produces forecasts at horizon h, i.e., we have forecasts s^h(t) for all t that depend deterministically on all s(t′) with t′≤t−h. Write rh(t):=s(t)−s^h(t) for the forecast residuals at horizon h.
> 
> Note these are all residuals _at_ a forecast horizon h. Often it is assumed (and let's do that for simplicity) that rh(t) is variance stationary, so Var rh(t) does not depend on t, but it depends on h, so let's call this σh2.
> 
> Let's say we have these residuals at time stamps t∈R. The typical estimate for σh2 would be 1|R|⋅ρh∑t∈R(rh(t)−r¯h)2, where r¯h is the sample mean of the rh(t), ranging over t∈R, and ρh is effective sample size correction coefficient of rh (this can be estimated in various ways).
> 
> What is important is not how we compute ρh (there are different ways), but the sum we need to compute (which does not depend on ρh).
> 
> For the naive forecaster, the sum would be computed from `y - y.shift(h)` for "last value carry forward" (in the non-seasonal case), and we get one σh2 per h in the `sktime` forecasting horizon vector `fh`. Similar for rolling mean etc.
> 
> Which is not what the current code does: it does not loop over h-es, and it does not look at the right residuals.

I still need more time to digest your point about the `last` method.

But I also think that your point to make drift rolling for non-null `window_length` doesn't seem quite right to me intuitively. If we compute residuals in this way, instead of just computing the residuals relative to a SINGLE line, the "naive" drift model is assumption a more complex model for the time series: i.e. a piecewise approximation to a polynomial regression.
> In my mind, a "mean" naive strategy is fundamentally different to a "moving average" forecaster

But would that be a mistake in impementation, or simply a different choice of naive method?

If it were spelled out in the docstring and said explicitly what mean means (pun intended), I would see no problem with assuming a moving average model.

In fact, the windowed moving average model seems more reasonable to assume than just a constant plus i.i.d. shocks.
> If we compute residuals in this way, instead of just computing the residuals relative to a SINGLE line, the "naive" drift model becomes something quite complex: a piecewise approximation to a polynomial regresion...

I think the *choice* of naive method is not a point, there are many choices here, why not have many (mean overall, mean rolling, etc).

The point is how, given that choice, the predictive variance is computed. Whichever the choice, there should be (a) properly computed rolling residuals and (b) a loop over forecasting horizon elements. In my current opinion, or it's not correct.
Side note, not important but to point out: the "mean over the entire history" is a special case of rolling mean with infinite (or very large) window length.
I think what is for certain is the assumptions and implementation behind sktime's `NaiveForecaster` goes far ~behind~ beyond the canomial naive methods as outlined in FPP3. 

> > In my mind, a "mean" naive strategy is fundamentally different to a "moving average" forecaster
> 
> But would that be a mistake in impementation, or simply a different choice of naive method?
> 
> In fact, the windowed moving average model seems more reasonable to assume than just a constant plus i.i.d. shocks.

In my opinion, after reading deeper into the literature, it was a mistake for `sktime` to have introduced `window_length`. Moving average is covered by the `ARIMA(0,0,K)` model. And perhaps you are right that the "MA" model is more reasonable for many cases than the constant mean, but there physical systems in the world that behave in the way that "straight line" mean with IID shocks model assumes. For example, any physical system in an unstable equillibrium.

> Side note, not important but to point out: the "mean over the entire history" is a special case of rolling mean with infinite (or very large) window length.

This is true, but many things in statistics and mathematics can be expressed as special cases of other equations and theorems. However, as applied statisticians, it is important to note that some special cases can be different enough (in terms of what they imply about our view of the world) to be considered as a separate "model".

IMO this "infinite moving average" / straight line with iid shocks is one such special case.
> Given the agreement with the R version, this will probably mean that either none of the two implementations have buggy math, or both. The latter is acceptable as a temporary state since `forecast` is a "reference" package.

@jasmineliaw what you linked should be identical to `fable` right?


> > Given the agreement with the R version, this will probably mean that either none of the two implementations have buggy math, or both. The latter is acceptable as a temporary state since `forecast` is a "reference" package.
> 
> @jasmineliaw what you linked should be identical to `fable` right?

Yes, `fable` did the same thing.

- https://github.com/tidyverts/fable/blob/master/R/lagwalk.R#L26

```R
fitted <- c(rep(NA, min(lag, n)), utils::head(fits, -lag))
```

- https://github.com/tidyverts/fable/blob/master/R/lagwalk.R#L45
```R
res <- y - fitted
```
> > If we compute residuals in this way, instead of just computing the residuals relative to a SINGLE line, the "naive" drift model becomes something quite complex: a piecewise approximation to a polynomial regresion...
> 
> I think the _choice_ of naive method is not a point, there are many choices here, why not have many (mean overall, mean rolling, etc).
> 
> The point is how, given that choice, the predictive variance is computed. Whichever the choice, there should be (a) properly computed rolling residuals and (b) a loop over forecasting horizon elements. In my current opinion, or it's not correct.

I am all for your idea of computing residuals across many windows (i.e. in the spirit of cross validation), but the algorithms and equations that PR3435 and R implements relies on strong parametric assumptions about the time series's data generating process for the population (past, present, future). For better or for worse...

What you are suggesting (which I use in practice for all forecasting workflows) is more in the tradition of bootstrap or even conformal intervals, but I see it as a fundamentally different model of the underyling DGP.

@fkiraly Please let me know if I'm not quite right about this point
> @fkiraly Please let me know if I'm not quite right about this point

I think you might be missing the point?

What I am spelling out is just a standard way to estimate the forecast variance as far as I know.
Also see the Hyndman book: https://otexts.com/fpp3/prediction-intervals.html

The $e_t$ in the book are equal to the $r_1(t)$ in my post above. The $e_t$ are not defined directly there but in the preceding section(s)
https://otexts.com/fpp3/diagnostics.html

and they correspond to what I said, and not, e.g., the last value overall for the "last value carry forward" method.

Spelling out my math for that forecaster gives $r(t) = y(t) - y(t-1)$ (as in the book, section 5.4); your code - and therefore, presumably, the R code, too - gives $r(t) = y(t) - y(T)$, which seems wrong.
@fkiraly I agree with your assessment for the `last` method. But I think r(t)=y(t)−y(t−1), which I assume you got here:
<img width="859" alt="Screenshot 2022-09-24 001907" src="https://user-images.githubusercontent.com/46541035/192006832-20e0ef45-fb57-4426-aa81-b1246a85b57c.png">
only refers to the equation for the residuals for the `last` AKA first differences method of the naive forecaster. I can't see any point in that section of the textbook that suggests the "mean" forecaster and "drift" forecasters have their residuals computed using this "first differences" equation. Nor does it suggest all residuals for every forecaster are estimated in that way. In my mind, the most general equation for "residuals" is this:
<img width="850" alt="Screenshot 2022-09-24 002655" src="https://user-images.githubusercontent.com/46541035/192008180-531907b8-982d-4daa-a2cf-27aa7153d0fd.png">
(https://otexts.com/fpp3/residuals.html#residuals-1)
where $\hat{y}_t$ is a sample estimate of $y_t$. There is no notation that suggests $\hat{y}_t$ needs to have any dependence with $t-k$ for any $k$.

I think what the R code has (and what I have) relies heavily on the **homogeneity and no autocorrelation assumptions** of the errors, so the residuals are computed as if the whole $y$ series is some cross-sectional dataset.

We should probably clarify what we mean by "naive"! We seem to be covering too many different models into this one forecaster.
> only refers to the equation for the residuals for the last AKA first differences method of the naive forecaster.

Yes, I was using it as an example only.

In the context of section 5.5, in general, $\widehat{y}_t$ needs to be the forecast made from cutoff $t-1$ at horizon $h=1$ ,

i.e., after applying the forecasting algorithm to the partial series $y_1, \dots, y_{t-1}$.

So, another example: if you are forecasting the mean of the series observed, you would have to take $\widehat{y}_t= \frac{1}{t-1} \sum^{t-1}_{i=1} y_i,$

and not $\widehat{y}_t = \frac{1}{T} \sum_{i=1}^{T} y_i$

(using the notation from the Hyndman book throughout)
something isn't going well with the TeX renderer here. probably the underscores.

Using bracket notation, $y(t)$ etc, the above would read:

In the context of section 5.5, in general, $\widehat{y}(t)$ needs to be the forecast made from cutoff $t-1$ at horizon $h=1$ ,

i.e., after applying the forecasting algorithm to the partial series $y(1), \dots, y(t-1)$.

So, another example: if you are forecasting the mean of the series observed, you would have to take $\widehat{y}(t)= \frac{1}{t-1} \sum_{i=1}^{t-1} y(i),$

and not $\widehat{y}(t) = \frac{1}{T} \sum_{i=1}^{T} y(i)$
It makes sense but I don't think it's what the R code assumes. I think the R implementation assumes the residuals are the distance between the observed y and the fitted y (fitted on the full historical dataset), without any concept of "forecasting" or data spillage. Happy to follow your interpretation but the R comparison test will break.
@fkiraly How do you suggest we move forward?
Hm. Is it documented somewhere what the R code assumes?

The answer might be simple, there is a bug in both pieces of code?
We could ask one of its authors.

Either way, I also think we ought to write up somehwere what the methods do, or at least, what we think it should do.
I feel the above discussion has made clear that a reference to a section of Hyndman is not precise enough, since there are too many gaps to fill in.
> Hm. Is it documented somewhere what the R code assumes?
> 
@fkiraly Agreed that we need a more detailed breakdown of what each method does. Especially because "naive" methods are often **extremely** hard to beat and deserve close attention. Here are my findings after a deeper deep dive of the R code.

#### Question: How are residuals being computed for each "naive" method in R [fable](https://github.com/tidyverts/fable)?
A) `last` AKA `lag` model in R fable's [`lagwalk` module](https://github.com/tidyverts/fable/blob/master/R/lagwalk.R)
```R
fits <- stats::lag(y, -lag)  # line 19
sigma <- stats::sd(y - fitted, na.rm = TRUE)   # line 44
res <- y - fitted  # line   # line 45
```
**Implication:** R fable computes residuals against a single window of observed y. `fh` does not affect prediction standard errors / variance.

B) `mean` AKA `mean` model in R fable's [`mean` module](https://github.com/tidyverts/fable/blob/master/R/mean.R)
```R
  # Lines 13-28
  n <- length(y)
  window_size <- specials$window[[1]]
  # Case 1. mean of entire training `y`
  if (is.null(window_size)) {
    y_mean <- mean(y, na.rm = TRUE)
    fits <- rep(y_mean, n)  # equivalent to np.repeat in numpy
  }
  # Case 2. rolling mean
  else {
    fits <- slide_dbl(y, mean,
      na.rm = TRUE,
      .size = window_size, .partial = TRUE
    )  # Compute rolling mean
    y_mean <- fits[length(fits)]
    fits <- dplyr::lag(fits)  # equivalent to shift operation in pandas
  }
  res <- y - fits
  sigma <- sd(res, na.rm = TRUE)
```
**Implication:** For both null and non-null `window_length`, R fable computes residuals against a single window of observed y. `fh` does not affect prediction standard errors / variance.

C) `drift` AKA `drift` model in R fable's [`lagwalk` module](https://github.com/tidyverts/fable/blob/master/R/lagwalk.R)
```R
    fitted  <- c(rep(NA, min(lag, n)), utils::head(fits, -lag))  # Line 26
    # Lines 28 - 39
    fit <- summary(stats::lm(y - fitted ~ 1, na.action = stats::na.exclude))
    b <- fit$coefficients[1, 1]
    b.se <- fit$coefficients[1, 2]
    fitted <- fitted + b
  ...
  # Lines 44 - 45
  sigma <- stats::sd(y - fitted, na.rm = TRUE)
  res <- y - fitted
```
**Implication:** R fable assumes the elements in the residuals vector are the distances between a single $y = mx + c$ line and the observed y. `fh` does not affect prediction standard errors / variance. Therefore, if `sktime` were to follow this algorithm for computing residuals for `drift`, the residuals (for any `window_length` and any `fh`) would be the vertical distances between a single drift line drawn between points `y[-(window_length or len(y))]` to `y[-1]` extrapolated *across the whole `y`* i.e. from `x=0` to `x=len(y)` and the observed y.
So I did notice 2 bugs in #3435, but if we were to follow R fable's algorithm for computing residuals for the "naive" methods: it would be a simple fix:
1. Replace `y_pred = np.repeat(np.squeeze(self.predict(fh=1)), T)` with:
```python
if not window_length:
  if sp > 1:
     y_pred = np.tile(self.predict(fh=list(range(sp)), math.ceil(T // sp))[0:T]
  else:
     y_pred = np.repeat(np.squeeze(self.predict(fh=1)), T)
else:
  if sp > 1:
     seasons = np.mod(np.arange(T), sp)
     y_pred = (
        y.to_frame()
        .assign(__sp__=seasons)
        .groupby("__sp__")
        .rolling(window_length)
        .mean()
        .droplevel("__sp__")
        .sort_index()
        .squeeze()
     )
  else:
     y_pred = y.rolling(window_length).mean()
``` 
2. Replace `slope = (y.iloc[-1] - y.iloc[0]) / (T - 1)` with `slope = (y.iloc[-1] - y.iloc[-(window_length or 0)]) / (window_length - 1)` 

Buggy lines can be found here:
https://github.com/topher-lo/sktime/blob/61708e78c02a29570bd084c82933ef690508eac9/sktime/forecasting/naive.py#L433)
https://github.com/topher-lo/sktime/blob/61708e78c02a29570bd084c82933ef690508eac9/sktime/forecasting/naive.py#L438)


Resolving patch:
diff --git a/sktime/forecasting/naive.py b/sktime/forecasting/naive.py
--- a/sktime/forecasting/naive.py
+++ b/sktime/forecasting/naive.py
@@ -15,6 +15,7 @@
     "bethrice44",
 ]
 
+import math
 from warnings import warn
 
 import numpy as np
@@ -23,6 +24,7 @@
 
 from sktime.datatypes._convert import convert, convert_to
 from sktime.datatypes._utilities import get_slice
+from sktime.forecasting.base import ForecastingHorizon
 from sktime.forecasting.base._base import DEFAULT_ALPHA, BaseForecaster
 from sktime.forecasting.base._sktime import _BaseWindowForecaster
 from sktime.utils.validation import check_window_length
@@ -423,23 +425,60 @@ def _predict_var(self, fh, X=None, cov=False):
         y = self._y
         y = convert_to(y, "pd.Series")
         T = len(y)
+        sp = self.sp
 
         # Compute "past" residuals
         if self.strategy == "last":
             y_res = y - y.shift(self.sp)
         elif self.strategy == "mean":
-            # Since this strategy returns a constant, just predict fh=1 and
-            # transform the constant into a repeated array
-            y_pred = np.repeat(np.squeeze(self.predict(fh=1)), T)
+            if not self.window_length:
+                if sp > 1:
+                    # Get the next sp predictions and repeat them
+                    # T / self.sp times to match the length of trained y
+                    # NOTE: +1 extra tile to defend against off-by-one errors
+                    reps = math.ceil(T / sp) + 1
+                    past_fh = ForecastingHorizon(
+                        list(range(1, sp + 1)), is_relative=None, freq=self._cutoff
+                    )
+                    seasonal_means = self._predict(fh=past_fh)
+                    if isinstance(seasonal_means, pd.DataFrame):
+                        seasonal_means = seasonal_means.squeeze()
+                    y_pred = np.tile(seasonal_means.to_numpy(), reps)[0:T]
+                else:
+                    # Since this strategy returns a constant, just predict fh=1 and
+                    # transform the constant into a repeated array
+                    past_fh = ForecastingHorizon(1, is_relative=None, freq=self._cutoff)
+                    y_pred = np.repeat(np.squeeze(self._predict(fh=past_fh)), T)
+            else:
+                if sp > 1:
+                    # Label index by seasonal period
+                    seasons = np.mod(np.arange(T), sp)
+                    # Compute rolling seasonal means
+                    y_pred = (
+                        y.to_frame()
+                        .assign(__sp__=seasons)
+                        # Group observations by their
+                        # seasonal period position
+                        .groupby("__sp__")
+                        # Compute rolling means per
+                        # seasonal period
+                        .rolling(self.window_length)
+                        .mean()
+                        .droplevel("__sp__")
+                        .sort_index()
+                        .squeeze()
+                    )
+                else:
+                    # Compute rolling means
+                    y_pred = y.rolling(self.window_length).mean()
             y_res = y - y_pred
         else:
             # Slope equation from:
             # https://otexts.com/fpp3/simple-methods.html#drift-method
-            slope = (y.iloc[-1] - y.iloc[0]) / (T - 1)
-
+            slope = (y.iloc[-1] - y.iloc[-(self.window_length or 0)]) / (T - 1)
             # Fitted value = previous value + slope
             # https://github.com/robjhyndman/forecast/blob/master/R/naive.R#L34
-            y_res = y - (y.shift(self.sp) + slope)
+            y_res = y - (y.shift(sp) + slope)
 
         # Residuals MSE and SE
         # + 1 degrees of freedom to estimate drift coefficient standard error
@@ -448,7 +487,6 @@ def _predict_var(self, fh, X=None, cov=False):
         mse_res = np.sum(np.square(y_res)) / (T - n_nans - (self.strategy == "drift"))
         se_res = np.sqrt(mse_res)
 
-        sp = self.sp
         window_length = self.window_length or T
         # Formulas from:
         # https://otexts.com/fpp3/prediction-intervals.html (Table 5.2)


Test patch:
diff --git a/sktime/forecasting/tests/test_naive.py b/sktime/forecasting/tests/test_naive.py
--- a/sktime/forecasting/tests/test_naive.py
+++ b/sktime/forecasting/tests/test_naive.py
@@ -267,11 +267,13 @@ def test_strategy_mean_and_last_seasonal_additional_combinations(
         ("mean", 1, None),
         ("mean", 24, None),
         ("mean", 1, 24),
-        ("mean", 24, 24),
+        ("mean", 24, 48),
         ("drift", 1, None),
+        ("drift", 1, 24),
     ],
 )
-def test_naive_predict_var_backwards(strategy, sp, window_length):
+@pytest.mark.parametrize("n_periods", [10000, 9999, 12128])
+def test_naive_predict_var_backwards(strategy, sp, window_length, n_periods):
     """Checks naive prediction variance computations.
 
     Test whether, given h=1 and large T, the forecast standard error
@@ -289,11 +291,10 @@ def test_naive_predict_var_backwards(strategy, sp, window_length):
     ----------
     .. [1] https://otexts.com/fpp3/prediction-intervals.html#benchmark-methods
     """
-    n_timepoints = 100000
     mu, sigma = 0.0, 10.0
-    fake_idx = pd.date_range("1980", periods=n_timepoints + 1, freq="H")
+    fake_idx = pd.date_range("1980", periods=n_periods + 1, freq="H")
     np.random.seed(42)
-    y = pd.Series(np.random.normal(mu, sigma, size=n_timepoints), index=fake_idx[:-1])
+    y = pd.Series(np.random.normal(mu, sigma, size=n_periods), index=fake_idx[:-1])
 
     h = 1
     forecaster = NaiveForecaster(strategy, sp=sp, window_length=window_length)
@@ -313,6 +314,38 @@ def test_naive_predict_var_backwards(strategy, sp, window_length):
     assert lower_bound < sigma_res < upper_bound
 
 
+@pytest.mark.parametrize(
+    "strategy,sp,window_length",
+    [
+        ("last", 1, None),
+        ("last", 24, None),
+        ("mean", 1, None),
+        ("mean", 24, None),
+        ("mean", 1, 24),
+        ("mean", 24, 48),
+        ("drift", 1, None),
+        ("drift", 1, 24),
+    ],
+)
+@pytest.mark.parametrize("fh", TEST_OOS_FHS)
+def test_naive_predict_interval_mean(strategy, sp, window_length, fh):
+    """Checks naive prediction interval means are equal to mean predictions.
+
+    Note: this is largely a smoke test to check the validity of tricky rolling
+    mean and seasonal mean array operations within pred_var.
+    """
+    n_timepoints = 100000
+    mu, sigma = 0.0, 10.0
+    fake_idx = pd.date_range("1980", periods=n_timepoints + 1, freq="H")
+    np.random.seed(42)
+    y = pd.Series(np.random.normal(mu, sigma, size=n_timepoints), index=fake_idx[:-1])
+
+    forecaster = NaiveForecaster(strategy, sp=sp, window_length=window_length).fit(y)
+    y_pred_interval = forecaster.predict_interval(fh=fh)
+    y_pred = forecaster.predict(fh=fh)
+    pd.testing.assert_series_equal(y_pred_interval.mean(axis=1), y_pred)
+
+
 @pytest.mark.parametrize(
     "strategy,sp,lower,upper",
     [