Repository: alan-turing-institute/sktime
Pull request #2833 (2022-06-19 08:31:10)

[ENH] Reducer prototype rework - experimental
Prototype rework of the direct reducer, to showcase how a larger refactor could work and solicit feedback.

Reworks the direct reducer in order to:
* allow concurrent use of exogeneous data `X`, up to and including the setting where only `X` at the same time is used to predict `y`
* support `pandas` input to `sklearn` estimators and associated features, e.g., inspecting fitted model parameters with feature names
* factor out the complexity of lagging and windowing to the transformer `Lag`, and of imputing to the transformer `Imputer`. The code logic specific to the redicuer
* support for time stamp-like lags, through `Lag`

Depends on fix https://github.com/alan-turing-institute/sktime/pull/2832 since `ForecastingHorizon` yields numpy integers which apparently break the `Lag` transformer.

FYI @grzegorzrut, @danbartl

fkiraly: FYI @kishmanani
KishManani: Correct me if I'm wrong, but, with this implementation you specify whether you want you want all exogenous features to be fitted concurrently (i.e., we know the values of the features in the future) or shifted (i.e., we might use this in most cases when we don't know the value of the features in the future). In practice, we'll have a mix of these types of features. I think this is one reason why the Darts API distinguishes between, in the language of Darts, future covariates (when you know the future value of features) and past covariates (where you only know the past value of features and hence tend to use lagged versions of them). 
fkiraly: > Correct me if I'm wrong, but, with this implementation you specify whether you want you want all exogenous features to be fitted concurrently

Yes - all the exogeneous features *that reach this estimator*.

You can deal by that by composition. I.e., you use the concurrent reducer, but before that you apply a `ColumnTransformer` with a suitable `Lag` applied to the "past features" only. Would that not work?

Or, would you want a mix, i.e., past features being "shifted", "future" being concurrent? For a single element in the forecasting horizon, you can still get that.

Alternatively, we could add arguments that specify "hidden" and "shifted" features?

> In practice, we'll have a mix of these types of features. I think this is one reason why the Darts API distinguishes between, in the language of Darts, future covariates (when you know the future value of features) and past covariates (where you only know the past value of features and hence tend to use lagged versions of them).

I feel there is a problem with distinguishing the "past" and "future" covariates in the API, namely that your composition options and syntax proliferate, i.e., when you build pipelines.

Do you want to apply something to "past" or "future" or "target? How do you specify cleanly in a pipeline if you want to apply a transformer to one and move it across to the other? It is already difficult to cover the case where you separate into "endogeneous" and "exogeneous" by separating arguments.

My current conceptualization is, from the data you already *know* which features are "past" or "future" - the ones which are observed only in the "past" are, well, "past". You do not need to specify this in addition, or even manually separate your data.

If you want to deal with these separately (and often you do), you can use `ColumnTransformer`.

Would that not always work?

Or do you have examples where it is more convenient, in composition, to deal with them separately interface-wise?

Patch:
diff --git a/sktime/forecasting/base/_fh.py b/sktime/forecasting/base/_fh.py
index 660ecabd7..cf422c3a0 100644
--- a/sktime/forecasting/base/_fh.py
+++ b/sktime/forecasting/base/_fh.py
@@ -121,13 +121,15 @@ def _check_values(values: Union[VALID_FORECASTING_HORIZON_TYPES]) -> pd.Index:
     else:
         valid_types = (
             "int",
-            "np.array",
+            "1D np.ndarray of type int",
+            "1D np.ndarray of type timedelta or dateoffset",
             "list",
             *[f"pd.{index_type.__name__}" for index_type in VALID_INDEX_TYPES],
         )
         raise TypeError(
             f"Invalid `fh`. The type of the passed `fh` values is not supported. "
-            f"Please use one of {valid_types}, but found: {type(values)}"
+            f"Please use one of {valid_types}, but found type {type(values)}, "
+            f"values = {values}"
         )
 
     # check values does not contain duplicates
diff --git a/sktime/forecasting/compose/_reduce.py b/sktime/forecasting/compose/_reduce.py
index 5cf055cb1..3ec20ac87 100644
--- a/sktime/forecasting/compose/_reduce.py
+++ b/sktime/forecasting/compose/_reduce.py
@@ -5,11 +5,12 @@
 """Composition functionality for reduction approaches to forecasting."""
 
 __author__ = [
+    "mloning",
     "AyushmaanSeth",
     "Kavin Anand",
     "Luis Zugasti",
-    "Lovkush Agarwal",
-    "Markus Löning",
+    "Lovkush-A",
+    "fkiraly",
 ]
 
 __all__ = [
@@ -22,14 +23,18 @@ __all__ = [
     "MultioutputTabularRegressionForecaster",
     "DirRecTabularRegressionForecaster",
     "DirRecTimeSeriesRegressionForecaster",
+    "DirectReductionForecaster",
 ]
 
+from warnings import warn
+
 import numpy as np
 import pandas as pd
 from sklearn.base import RegressorMixin, clone
+from sklearn.multioutput import MultiOutputRegressor
 
 from sktime.datatypes._utilities import get_time_index
-from sktime.forecasting.base import ForecastingHorizon
+from sktime.forecasting.base import BaseForecaster, ForecastingHorizon
 from sktime.forecasting.base._base import DEFAULT_ALPHA
 from sktime.forecasting.base._fh import _index_range
 from sktime.forecasting.base._sktime import _BaseWindowForecaster
@@ -1246,3 +1251,313 @@ def _create_fcst_df(target_date, origin_df, fill=None):
 
         template = template.astype(origin_df.dtypes.to_dict())
         return template
+
+
+def _coerce_col_str(X):
+    """Coerce columns to string, to satisfy sklearn convention."""
+    X.columns = [str(x) for x in X.columns]
+    return X
+
+
+class DirectReductionForecaster(BaseForecaster):
+    """Direct reduction forecaster, incl single-output, multi-output, exogeneous Dir.
+
+    Implements direct reduction, of forecasting to tabular regression.
+
+    For no `X`, defaults to DirMO (direct multioutput) for `X_treatment = "concurrent"`,
+    and simple direct (direct single-output) for `X_treatment = "shifted"`.
+
+    Direct single-output with concurrent `X` behaviour can be configured
+    by passing a single-output `scikit-learn` compatible transformer.
+
+    Algorithm details:
+
+    In `fit`, given endogeneous time series `y` and possibly exogeneous `X`:
+        fits `estimator` to feature-label pairs as defined as follows.
+    if `X_treatment = "concurrent":
+        features = `y(t)`, `y(t-1)`, ..., `y(t-window_size)`, if provided: `X(t+h)`
+        labels = `y(t+h)` for `h` in the forecasting horizon
+        ranging over all `t` where the above have been observed (are in the index)
+        for each `h` in the forecasting horizon (separate estimator fitted per `h`)
+    if `X_treatment = "shifted":
+        features = `y(t)`, `y(t-1)`, ..., `y(t-window_size)`, if provided: `X(t)`
+        labels = `y(t+h_1)`, ..., `y(t+h_k)` for `h_j` in the forecasting horizon
+        ranging over all `t` where the above have been observed (are in the index)
+        estimator is fitted as a multi-output estimator (for all  `h_j` simultaneously)
+
+    In `predict`, given possibly exogeneous `X`, at cutoff time `c`,
+    if `X_treatment = "concurrent":
+        applies fitted estimators' predict to
+        feature = `y(c)`, `y(c-1)`, ..., `y(c-window_size)`, if provided: `X(c+h)`
+        to obtain a prediction for `y(c+h)`, for each `h` in the forecasting horizon
+    if `X_treatment = "shifted":
+        applies fitted estimator's predict to
+        features = `y(c)`, `y(c-1)`, ..., `y(c-window_size)`, if provided: `X(t)`
+        to obtain prediction for `y(c+h_1)`, ..., `y(c+h_k)` for `h_j` in forec. horizon
+
+    Parameters
+    ----------
+    estimator : sklearn regressor, must be compatible with sklearn interface
+        tabular regression algorithm used in reduction algorithm
+    window_length : int, optional, default=10
+        window length used in the reduction algorithm
+    transformers : currently not used
+    X_treatment : str, optional, one of "concurrent" (default) or "shifted"
+        determines the timestamps of X from which y(t+h) is predicted, for horizon h
+        "concurrent": y(t+h) is predicted from lagged y, and X(t+h), for all h in fh
+            in particular, if no y-lags are specified, y(t+h) is predicted from X(t)
+        "shifted": y(t+h) is predicted from lagged y, and X(t), for all h in fh
+            in particular, if no y-lags are specified, y(t+h) is predicted from X(t+h)
+    impute : str or None, optional, method string passed to Imputer
+        default="bfill", admissible strings are of Imputer.method parameter, see there
+        if None, no imputation is done when applying Lag transformer to obtain inner X
+    """
+
+    _tags = {
+        "requires-fh-in-fit": True,  # is the forecasting horizon required in fit?
+        "X_inner_mtype": "pd.DataFrame",
+        "y_inner_mtype": "pd.DataFrame",
+    }
+
+    def __init__(
+        self,
+        estimator,
+        window_length=10,
+        transformers=None,
+        X_treatment="concurrent",
+        impute_method="bfill",
+    ):
+        self.window_length = window_length
+        self.transformers = transformers
+        self.transformers_ = None
+        self.estimator = estimator
+        self.X_treatment = X_treatment
+        self.impute_method = impute_method
+        self._lags = list(range(window_length))
+        super(DirectReductionForecaster, self).__init__()
+
+        warn(
+            "DirectReductionForecaster is experimental, and interfaces may change. "
+            "user feedback is appreciated in issue #3224 here: "
+            "https://github.com/alan-turing-institute/sktime/issues/3224"
+        )
+
+    def _fit(self, y, X=None, fh=None):
+        """Fit dispatcher based on X_treatment."""
+        methodname = f"_fit_{self.X_treatment}"
+        return getattr(self, methodname)(y=y, X=X, fh=fh)
+
+    def _predict(self, X=None, fh=None):
+        """Predict dispatcher based on X_treatment."""
+        methodname = f"_predict_{self.X_treatment}"
+        return getattr(self, methodname)(X=X, fh=fh)
+
+    def _fit_shifted(self, y, X=None, fh=None):
+        """Fit to training data."""
+        from sktime.transformations.series.impute import Imputer
+        from sktime.transformations.series.lag import Lag
+
+        impute_method = self.impute_method
+
+        # lagger_y_to_X_ will lag y to obtain the sklearn X
+        lags = self._lags
+        lagger_y_to_X = Lag(lags=lags, index_out="extend")
+        if impute_method is not None:
+            lagger_y_to_X = lagger_y_to_X * Imputer(method=impute_method)
+        self.lagger_y_to_X_ = lagger_y_to_X
+
+        # lagger_y_to_y_ will lag y to obtain the sklearn y
+        fh_rel = fh.to_relative(self.cutoff)
+        y_lags = list(fh_rel)
+        y_lags = [-x for x in y_lags]
+        lagger_y_to_y = Lag(lags=y_lags, index_out="original")
+        self.lagger_y_to_y_ = lagger_y_to_y
+
+        yt = lagger_y_to_y.fit_transform(y)
+        y_notna = yt.notnull().all(axis=1)
+        y_notna_idx = y_notna.index[y_notna]
+
+        # we now check whether the set of full lags is empty
+        # if yes, we set a flag, since we cannot fit the reducer
+        # instead, later, we return a dummy prediction
+        if len(y_notna_idx) == 0:
+            self.empty_lags_ = True
+            self.dummy_value_ = y.mean()
+            return self
+        else:
+            self.empty_lags_ = False
+
+        yt = yt.loc[y_notna_idx]
+        Xt = lagger_y_to_X.fit_transform(y).loc[y_notna_idx]
+
+        if X is not None:
+            Xt = pd.concat([X.loc[y_notna_idx], Xt], axis=1)
+
+        Xt = _coerce_col_str(Xt)
+        yt = _coerce_col_str(yt)
+
+        estimator = clone(self.estimator)
+        if not estimator._get_tags()["multioutput"]:
+            estimator = MultiOutputRegressor(estimator)
+        estimator.fit(Xt, yt)
+        self.estimator_ = estimator
+
+        return self
+
+    def _predict_shifted(self, fh=None, X=None):
+        """Predict core logic."""
+        fh_idx = pd.Index(fh.to_absolute(self.cutoff))
+        y_cols = self._y.columns
+
+        if self.empty_lags_:
+            ret = pd.DataFrame(index=fh_idx, columns=y_cols)
+            for i in X.index:
+                X.loc[i] = self.dummy_value_
+            return ret
+
+        lagger_y_to_X = self.lagger_y_to_X_
+
+        Xt_lastrow = lagger_y_to_X.transform(self._y).loc[[self.cutoff]]
+        if self._X is not None:
+            Xt_lastrow = pd.concat([self._X.loc[[self.cutoff]], Xt_lastrow], axis=1)
+
+        Xt_lastrow = _coerce_col_str(Xt_lastrow)
+
+        estimator = self.estimator_
+        # 2D numpy array with col index = (fh, var) and 1 row
+        y_pred = estimator.predict(Xt_lastrow)
+        y_pred = y_pred.reshape((len(fh), len(y_cols)))
+
+        y_pred = pd.DataFrame(y_pred, columns=y_cols, index=fh_idx)
+
+        return y_pred
+
+    def _fit_concurrent(self, y, X=None, fh=None):
+        """Fit to training data."""
+        from sktime.transformations.series.impute import Imputer
+        from sktime.transformations.series.lag import Lag
+
+        impute_method = self.impute_method
+
+        # lagger_y_to_X_ will lag y to obtain the sklearn X
+        lags = self._lags
+        lagger_y_to_X = Lag(lags=lags, index_out="extend")
+        if impute_method is not None:
+            lagger_y_to_X = lagger_y_to_X * Imputer(method=impute_method)
+        self.lagger_y_to_X_ = lagger_y_to_X
+
+        fh_rel = fh.to_relative(self.cutoff)
+        y_lags = list(fh_rel)
+
+        Xt = lagger_y_to_X.fit_transform(y)
+
+        self.estimators_ = []
+
+        for lag in y_lags:
+
+            lag_plus = Lag(lag, index_out="extend")
+            Xtt = lag_plus.fit_transform(Xt)
+            Xtt_notna = Xtt.notnull().all(axis=1)
+            Xtt_notna_idx = Xtt_notna.index[Xtt_notna].intersection(y.index)
+
+            yt = y.loc[Xtt_notna_idx]
+            Xtt = Xtt.loc[Xtt_notna_idx]
+
+            # we now check whether the set of full lags is empty
+            # if yes, we set a flag, since we cannot fit the reducer
+            # instead, later, we return a dummy prediction
+            if len(Xtt_notna_idx) == 0:
+                self.estimators_.append(y.mean())
+            else:
+                if X is not None:
+                    Xtt = pd.concat([X.loc[Xtt_notna_idx], Xtt], axis=1)
+
+                Xtt = _coerce_col_str(Xtt)
+                yt = _coerce_col_str(yt)
+
+                estimator = clone(self.estimator)
+                estimator.fit(Xtt, yt)
+                self.estimators_.append(estimator)
+
+        return self
+
+    def _predict_concurrent(self, X=None, fh=None):
+        """Fit to training data."""
+        from sktime.transformations.series.lag import Lag
+
+        if X is not None and self._X is not None:
+            X_pool = X.combine_first(self._X)
+        elif X is None and self._X is not None:
+            X_pool = self._X
+        else:
+            X_pool = X
+
+        fh_idx = pd.Index(fh.to_absolute(self.cutoff))
+        y_cols = self._y.columns
+
+        lagger_y_to_X = self.lagger_y_to_X_
+
+        fh_rel = fh.to_relative(self.cutoff)
+        fh_abs = fh.to_absolute(self.cutoff)
+        y_lags = list(fh_rel)
+        y_abs = list(fh_abs)
+
+        Xt = lagger_y_to_X.transform(self._y)
+        y_pred_list = []
+
+        for i in range(len(y_lags)):
+
+            lag = y_lags[i]
+            predict_idx = y_abs[i]
+
+            lag_plus = Lag(lag, index_out="extend")
+            Xtt = lag_plus.fit_transform(Xt)
+            Xtt_predrow = Xtt.loc[[predict_idx]]
+            if X_pool is not None:
+                Xtt_predrow = pd.concat(
+                    [X_pool.loc[[predict_idx]], Xtt_predrow], axis=1
+                )
+
+            Xtt_predrow = _coerce_col_str(Xtt_predrow)
+
+            estimator = self.estimators_[i]
+
+            # if = no training indices in _fit, fill in y training mean
+            if isinstance(estimator, pd.Series):
+                y_pred_i = pd.DataFrame(index=[0], columns=y_cols)
+                y_pred_i.iloc[0] = estimator
+            # otherwise proceed as per direct reduction algorithm
+            else:
+                y_pred_i = estimator.predict(Xtt_predrow)
+            # 2D numpy array with col index = (var) and 1 row
+            y_pred_list.append(y_pred_i)
+
+        y_pred = np.concatenate(y_pred_list)
+        y_pred = pd.DataFrame(y_pred, columns=y_cols, index=fh_idx)
+
+        return y_pred
+
+    @classmethod
+    def get_test_params(cls, parameter_set="default"):
+        """Return testing parameter settings for the estimator.
+
+        Parameters
+        ----------
+        parameter_set : str, default="default"
+            Name of the set of test parameters to return, for use in tests. If no
+            special parameters are defined for a value, will return `"default"` set.
+
+        Returns
+        -------
+        params : dict or list of dict, default = {}
+            Parameters to create testing instances of the class
+            Each dict are parameters to construct an "interesting" test instance, i.e.,
+            `MyClass(**params)` or `MyClass(**params[i])` creates a valid test instance.
+            `create_test_instance` uses the first (or only) dictionary in `params`
+        """
+        from sklearn.linear_model import LinearRegression
+
+        est = LinearRegression()
+        params1 = {"estimator": est, "window_length": 3, "X_treatment": "shifted"}
+        params2 = {"estimator": est, "window_length": 3, "X_treatment": "concurrent"}
+        return [params1, params2]
diff --git a/sktime/forecasting/model_selection/_split.py b/sktime/forecasting/model_selection/_split.py
index 9e23188b4..061e3d166 100644
--- a/sktime/forecasting/model_selection/_split.py
+++ b/sktime/forecasting/model_selection/_split.py
@@ -673,13 +673,16 @@ class CutoffSplitter(BaseSplitter):
     which using the notation provided in :class:`BaseSplitter`,
     can be written as :math:`(k_1,\ldots,k_n)` for integer based indexing,
     or :math:`(t(k_1),\ldots,t(k_n))` for datetime based indexing.
+
+    For a cutoff :math:`k_i` and a `window_length` :math:`w`
+    the training window is :math:`(k_i-w+1,k_i-w+2,k_i-w+3,\ldots,k_i)`.
     Training window's last point is equal to the cutoff.
 
     Test window is defined by forecasting horizons
     relative to the end of the training window.
     It will contain as many indices
     as there are forecasting horizons provided to the `fh` argument.
-    For a forecasating horizon :math:`(h_1,\ldots,h_H)`, the training window will
+    For a forecasating horizon :math:`(h_1,\ldots,h_H)`, the test window will
     consist of the indices :math:`(k_n+h_1,\ldots, k_n+h_H)`.
 
     The number of splits returned by `.get_n_splits`
@@ -690,7 +693,7 @@ class CutoffSplitter(BaseSplitter):
 
     Parameters
     ----------
-    cutoffs : list or np.array or pd.Index
+    cutoffs : list or np.ndarray or pd.Index
         Cutoff points, positive and integer- or datetime-index like.
         Type should match the type of `fh` input.
     fh : int, timedelta, list or np.ndarray of ints or timedeltas
@@ -724,34 +727,24 @@ class CutoffSplitter(BaseSplitter):
         window_length = check_window_length(
             window_length=self.window_length, n_timepoints=n_timepoints
         )
+        if isinstance(y, (pd.DatetimeIndex, pd.PeriodIndex)) and is_int(window_length):
+            window_length = y.freq * window_length
         _check_cutoffs_and_y(cutoffs=cutoffs, y=y)
         _check_cutoffs_fh_y(cutoffs=cutoffs, fh=fh, y=y)
-        max_fh = fh.max()
-        max_cutoff = np.max(cutoffs)
 
         for cutoff in cutoffs:
-            if is_int(x=window_length) and is_int(x=cutoff):
-                train_start = cutoff - window_length
-            elif is_timedelta_or_date_offset(x=window_length) and is_datetime(x=cutoff):
-                train_start = y.get_loc(max(y[0], cutoff - window_length))
+            null = 0 if is_int(cutoff) else pd.Timestamp(0)
+            if cutoff >= null:
+                train_end = y[cutoff] if is_int(cutoff) else cutoff
+                training_window = get_window(
+                    pd.Series(index=y[y <= train_end]), window_length=window_length
+                ).index
             else:
-                raise TypeError(
-                    f"Unsupported combination of types: "
-                    f"`window_length`: {type(window_length)}, "
-                    f"`cutoff`: {type(cutoff)}"
-                )
-
-            split_point = cutoff if is_int(x=cutoff) else y.get_loc(y[y <= cutoff][-1])
-            training_window = self._get_train_window(
-                y=y, train_start=train_start + 1, split_point=split_point + 1
-            )
-
+                training_window = []
+            training_window = y.get_indexer(training_window)
             test_window = cutoff + fh.to_numpy()
-            if is_datetime(x=max_cutoff) and is_timedelta(x=max_fh):
-                test_window = test_window[test_window >= y.min()]
-                test_window = np.array(
-                    [y.get_loc(timestamp) for timestamp in test_window]
-                )
+            if is_datetime(x=cutoff):
+                test_window = y.get_indexer(test_window[test_window >= y.min()])
             yield training_window, test_window
 
     def get_n_splits(self, y: Optional[ACCEPTED_Y_TYPES] = None) -> int:
diff --git a/sktime/forecasting/trend.py b/sktime/forecasting/trend.py
index a109e9c3b..082de42a2 100644
--- a/sktime/forecasting/trend.py
+++ b/sktime/forecasting/trend.py
@@ -15,7 +15,6 @@ from sklearn.preprocessing import PolynomialFeatures
 from statsmodels.tsa.seasonal import STL as _STL
 
 from sktime.forecasting.base import BaseForecaster
-from sktime.forecasting.naive import NaiveForecaster
 from sktime.utils.datetime import _get_duration
 
 
@@ -385,6 +384,8 @@ class STLForecaster(BaseForecaster):
         -------
         self : returns an instance of self.
         """
+        from sktime.forecasting.naive import NaiveForecaster
+
         self._stl = _STL(
             y.values,
             period=self.sp,
diff --git a/sktime/transformations/series/date.py b/sktime/transformations/series/date.py
index f4661312e..a2632b139 100644
--- a/sktime/transformations/series/date.py
+++ b/sktime/transformations/series/date.py
@@ -18,7 +18,7 @@ _RAW_DUMMIES = [
     ["quarter", "year", "quarter", "efficient"],
     ["month", "year", "month", "minimal"],
     ["week", "year", "week_of_year", "efficient"],
-    ["day", "year", "day", "efficient"],
+    ["day", "year", "day_of_year", "efficient"],
     ["month", "quarter", "month_of_quarter", "comprehensive"],
     ["week", "quarter", "week_of_quarter", "comprehensive"],
     ["day", "quarter", "day_of_quarter", "comprehensive"],
@@ -148,12 +148,12 @@ class DateTimeFeatures(BaseTransformer):
             if self.ts_freq is not None:
                 supported = _get_supported_calendar(self.ts_freq, DUMMIES=self.dummies)
                 supported = supported[supported["feature_scope"] <= self.feature_scope]
-                calendar_dummies = supported["dummy_func"].to_list()
+                calendar_dummies = supported[["dummy_func", "dummy"]]
             else:
                 supported = self.dummies[
                     self.dummies["feature_scope"] <= self.feature_scope
                 ]
-                calendar_dummies = supported["dummy_func"].to_list()
+                calendar_dummies = supported[["dummy_func", "dummy"]]
         else:
             if self.ts_freq is not None:
                 supported = _get_supported_calendar(self.ts_freq, DUMMIES=self.dummies)
@@ -164,16 +164,23 @@ class DateTimeFeatures(BaseTransformer):
                         "Level of selected dummy variable "
                         + " lower level than base ts_frequency."
                     )
-                calendar_dummies = supported.loc[
-                    supported["dummy"].isin(self.manual_selection), "dummy_func"
+                calendar_dummies = self.dummies.loc[
+                    self.dummies["dummy"].isin(self.manual_selection),
+                    ["dummy_func", "dummy"],
                 ]
             else:
                 calendar_dummies = self.dummies.loc[
-                    self.dummies["dummy"].isin(self.manual_selection), "dummy_func"
+                    self.dummies["dummy"].isin(self.manual_selection),
+                    ["dummy_func", "dummy"],
                 ]
 
-        df = [_calendar_dummies(x_df, dummy) for dummy in calendar_dummies]
+        df = [
+            _calendar_dummies(x_df, dummy) for dummy in calendar_dummies["dummy_func"]
+        ]
         df = pd.concat(df, axis=1)
+        df.columns = calendar_dummies["dummy"]
+        if self.manual_selection is not None:
+            df = df[self.manual_selection]
 
         Xt = pd.concat([Z, df], axis=1)
 
@@ -214,14 +221,11 @@ def _calendar_dummies(x, funcs):
         # calendar week of a year containing a Thursday.
         # So it is possible that a week in the new year is still
         # indexed starting in last year (week 52 or 53)
-        x[funcs] = date_sequence.isocalendar()["week"]
-        return x[funcs]
+        cd = date_sequence.isocalendar()["week"]
     elif funcs == "week_of_month":
-        x[funcs] = (date_sequence.day - 1) // 7 + 1
-        return x[funcs]
+        cd = (date_sequence.day - 1) // 7 + 1
     elif funcs == "month_of_quarter":
-        x[funcs] = (np.floor(date_sequence.month / 4) + 1).astype(np.int64)
-        return x[funcs]
+        cd = (np.floor(date_sequence.month / 4) + 1).astype(np.int64)
     elif funcs == "week_of_quarter":
         col_names = x.columns
         x_columns = col_names.intersection(["year", "quarter", "week"]).to_list()
@@ -239,11 +243,9 @@ def _calendar_dummies(x, funcs):
         ) - pd.tseries.offsets.QuarterBegin(startingMonth=1)
         df["qweek"] = df["qdate"].dt.isocalendar()["week"]
         df.loc[(df["quarter"] == 1) & (df["week"] < 52), "qweek"] = 0
-        df["week_of_quarter"] = df["week"] - df["qweek"] + 1
-        return df["week_of_quarter"]
+        cd = df["week"] - df["qweek"] + 1
     elif funcs == "millisecond":
-        x[funcs] = date_sequence.microsecond * 1000
-        return x[funcs]
+        cd = date_sequence.microsecond * 1000
     elif funcs == "day_of_quarter":
         quarter = date_sequence.quarter
         quarter_start = pd.DatetimeIndex(
@@ -255,11 +257,13 @@ def _calendar_dummies(x, funcs):
         values = (
             (x["date_sequence"] - quarter_start) / pd.to_timedelta("1D") + 1
         ).astype(int)
-        x[funcs] = values
-        return x[funcs]
+        cd = values
     else:
-        x[funcs] = getattr(date_sequence, funcs)
-        return x[funcs]
+        cd = getattr(date_sequence, funcs)
+    cd = pd.DataFrame(cd)
+    cd = cd.rename(columns={cd.columns[0]: funcs})
+    cd[funcs] = np.int64(cd[funcs])
+    return cd
 
 
 def _get_supported_calendar(ts_freq, DUMMIES):


Test patch:
diff --git a/sktime/transformations/series/tests/test_date.py b/sktime/transformations/series/tests/test_date.py
index c68530732..2d369fe05 100644
--- a/sktime/transformations/series/tests/test_date.py
+++ b/sktime/transformations/series/tests/test_date.py
@@ -64,6 +64,30 @@ pipe = DateTimeFeatures(manual_selection=["year", "second_of_minute"])
 pipe.fit(y_train)
 test_diffdateformat = pipe.transform(y_train).columns.to_list()
 
+pipe = DateTimeFeatures(ts_freq="L", feature_scope="comprehensive")
+pipe.fit(y_train)
+test_full = pipe.transform(y_train).columns.to_list()
+test_types = pipe.transform(y_train).select_dtypes(include=["int64"]).columns.to_list()
+
+all_args = [
+    "Number of airline passengers",
+    "year",
+    "quarter_of_year",
+    "month_of_year",
+    "week_of_year",
+    "day_of_year",
+    "month_of_quarter",
+    "week_of_quarter",
+    "day_of_quarter",
+    "week_of_month",
+    "day_of_month",
+    "day_of_week",
+    "hour_of_day",
+    "minute_of_hour",
+    "second_of_minute",
+    "millisecond_of_second",
+]
+
 
 @pytest.mark.parametrize(
     "test_input,expected",
@@ -77,8 +101,8 @@ test_diffdateformat = pipe.transform(y_train).columns.to_list()
                 "ARMED",
                 "POP",
                 "year",
-                "quarter",
-                "month",
+                "quarter_of_year",
+                "month_of_year",
                 "week_of_year",
                 "month_of_quarter",
                 "week_of_quarter",
@@ -87,7 +111,7 @@ test_diffdateformat = pipe.transform(y_train).columns.to_list()
         ),
         (
             test_reduced_featurescope,
-            ["GNPDEFL", "GNP", "UNEMP", "ARMED", "POP", "year", "month"],
+            ["GNPDEFL", "GNP", "UNEMP", "ARMED", "POP", "year", "month_of_year"],
         ),
         (
             test_changing_frequency,
@@ -98,18 +122,35 @@ test_diffdateformat = pipe.transform(y_train).columns.to_list()
                 "ARMED",
                 "POP",
                 "year",
-                "quarter",
-                "month",
+                "quarter_of_year",
+                "month_of_year",
                 "month_of_quarter",
             ],
         ),
-        (test_manspec_with_tsfreq, ["GNPDEFL", "GNP", "UNEMP", "ARMED", "POP", "year"]),
+        (
+            test_manspec_with_tsfreq,
+            ["GNPDEFL", "GNP", "UNEMP", "ARMED", "POP", "year", "second_of_minute"],
+        ),
         (
             test_manspec_wo_tsfreq,
-            ["GNPDEFL", "GNP", "UNEMP", "ARMED", "POP", "year", "second"],
+            ["GNPDEFL", "GNP", "UNEMP", "ARMED", "POP", "year", "second_of_minute"],
+        ),
+        (
+            test_univariate_data,
+            ["Number of airline passengers", "year", "second_of_minute"],
+        ),
+        (
+            test_diffdateformat,
+            ["Number of airline passengers", "year", "second_of_minute"],
+        ),
+        (
+            test_full,
+            all_args,
+        ),
+        (
+            test_types,
+            all_args[1:],
         ),
-        (test_univariate_data, ["Number of airline passengers", "year", "second"]),
-        (test_diffdateformat, ["Number of airline passengers", "year", "second"]),
     ],
 )
 def test_eval(test_input, expected):