Adding timeseries-tailored baseline strategy to Dummy* estimators. Making them more intelligent with strategy="best".

### Describe the workflow you want to enable

While always predicting the mean (for regression) or the most frequent class (for classification) are solid baselines for many ML workloads, they are too weak for time series problems, where the target is constantly changing. A much better, natural baseline in such cases is a lagged series value or the mean of the series over a recent window.

I would love to see a **Dummy strategy tailored for time series**, as sensible baselines are critical in these tasks to avoid optimistic bias.

Moreover, all Dummy estimators are currently simple and extremely fast, giving us room to add more intelligent strategies. What’s not very convenient in the current implementation is that it requires the user to manually test all simple strategies and choose the strongest to establish a reasonable baseline (which is something I always end up doing in practice).

I think we can relieve users of this burden by introducing a **"best" strategy that internally evaluates all existing strategies** and returns the best one based on the provided scoring method.

With these additions, we’ll make Dummy estimators smarter and more useful, capable of highlighting weak user solutions early on.

**Disadvantages:**

1)The typical runtime will increase from ~1/100th of a second to ~1/10th of a second, which I believe is negligible compared to the runtime of main models (usually minutes, if not hours);
2) A few new parameters added (namely, 4).

**Advantages:**

1) We’ll cover the important use case of time series, making baselines more realistic.
2) We’ll improve user experience by reducing the amount of code required, while offering stronger baselines with strategy="best".
3) No deprecation of parameters or breaking changes.

### Describe your proposed solution

0) We assume that the target y can either be a NumPy array or a Pandas Series with a datetime-like index.
1) We provide users the ability to specify a desired window when it's a time series problem. For that, we introduce the parameter **scope**: Literal["rolling", "expanding", "all"] = "all". If scope != "all", the strategy is applied not to the entire y series (in which case an ndarray is converted to a series), but to a window. A new parameter **kwargs**: dict = dict(window=1, closed="left") gets passed directly to the scope method of the y series.
To fully support weighted observations, we'll need an additional parameter: **extra_kwargs**: dict = {}.

```python
model = DummyRegressor(strategy="mean", scope="rolling", kwargs=dict(window=5, win_type="gaussian"), extra_kwargs=dict(std=0.1))
model.fit(X_train, y_train)

# Example processing
base = getattr(y_series, scope)(**self.kwargs)  # e.g., y_series.rolling(**self.kwargs)
# Supplementary SciPy arguments passed in the aggregation function
preds = getattr(base, self.strategy)(**self.extra_kwargs)  # e.g., y_series.rolling(**self.kwargs).mean(**self.extra_kwargs)

```
In kwargs, everything except min_periods, window, and win_type will revert to their default Pandas values (most crucially, closed is always set to "left" to avoid data leakage and an overly optimistic baseline).

2) We add the strategy "**lag**" (valid only when scope="rolling") which returns shifted target values. This is most suitable for regression but can also be used for classification.

3) We add the strategy "**ewm**" for the exponentially weighted moving average. In this case, kwargs gets passed directly to the y_series.ewm method.

4) To be flexible and fully consistent with Pandas' capabilities, we add the strategy "**apply**" (only valid when scope != "all"), allowing the user to specify their own custom aggregation function:

```python

def mad(x):
    return np.fabs(x - x.mean()).mean()

model = DummyRegressor(strategy="apply", scope="rolling", kwargs=dict(window="3D"), extra_kwargs=dict(func=mad, raw=True))
model.fit(X_train, y_train)

# Example processing
base = getattr(y_series, scope)(**self.kwargs)  # e.g., y_series.rolling(**self.kwargs)
preds = getattr(base, self.strategy)(**self.extra_kwargs)  # e.g., y_series.rolling(**self.kwargs).apply(**self.extra_kwargs)

```

5) We add the strategy "**best**", which internally runs all other available/feasible strategies and returns the best one (for the training set, of course). To determine what is "best," we add a new parameter, **scoring**, similar to the one in cross_validate. When evaluating the "rolling" strategy, for simplicity, only integer window indices are checked via np.correlate or, preferably, statsmodels' acf.

We'll have to think about ensuring that target is sorted (if it's a datetime-like indexed target), and about filling NAs (they can appear due to some windowing conditions). I think we can just backfill them, or maybe replace with results of the strategy run over entire dataset (ie, train set mean etc).

### Describe alternatives you've considered, if relevant

It all can be done in custom user code, of course. But doing it here will save writing/testing all these procedures for thousands of DS folks around the globe.

### Additional context

I can work on the PR if this idea will be approved.

COMMENT:
Just adding a bit more details to the answer of @jeremiedbb regarding the scope of scikit-learn. One result of working with time-series is that the API using `X` and `y` might not be the most adapted / be the most natural way to tackle some of the time-series problem like forecasting. This is one reason that we are uneasy to go towards this road. For instance, most package for time-series tends to change the API of `fit`, `predict`, etc.

Sometimes, we can manage to tackle some of the problems with only preprocessing (e.g. https://scikit-learn.org/dev/auto_examples/applications/plot_time_series_lagged_features.html#sphx-glr-auto-examples-applications-plot-time-series-lagged-features-py) but the total number of use cases that we can tackle naturally is probably not that much.

So we usually prefer to delegate the time-series estimators to specialized packages.

Labels: New Feature, Needs Decision - Close