Repository: sktime/sktime
Pull request: Add date time dummy extractor
<!--
Thanks for contributing a pull request! Please ensure you have taken a look
at our contribution guide: https://github.com/alan-turing-institute/sktime/blob/main/CONTRIBUTING.md
-->

#### Reference Issues/PRs
Closes https://github.com/alan-turing-institute/sktime/issues/1273
<!--
Example: Fixes #1234. See also #3456.

Please use keywords (e.g., Fixes) to create link to the issues or pull requests
you resolved, so that they will automatically be closed when your pull request
is merged. See https://github.com/blog/1506-closing-issues-via-pull-requests
-->


#### What does this implement/fix? Explain your changes.
This is a first draft.

Introduce method to generate dummy calender features
Will automatically generate flexible number of features based on the frequency of the series (provided as argument) and the complexity argument (ranges from 0 to 2) to represent few, many or all features.

#### Does your contribution introduce a new dependency? If yes, which one?

No

#### What should a reviewer concentrate their feedback on?

Scope of extractors, logic of complexity arguments

#### Any other comments?
Could not finalize Fourier Extractor before my vacation (in 2 weeks will continue on that).

#### PR checklist
<!--
Please go through the checklist below. Please feel free to remove points if they are not applicable.
-->

##### For all contributions
- [ ] I've added myself to the [list of contributors](https://github.com/alan-turing-institute/sktime/blob/main/.all-contributorsrc).
- [ ] Optionally, I've updated sktime's [CODEOWNERS](https://github.com/alan-turing-institute/sktime/blob/main/CODEOWNERS) to receive notifications about future changes to these files.
- [X] I've added unit tests and made sure they pass locally.

##### For new estimators
- [ ] I've added the estimator to the online documentation.
- [ ] I've updated the existing example notebooks or provided a new one to showcase how my estimator works.


<!--
Thanks for contributing!
-->


Problem statement:
[ENH] Calendar Feature Extractor
**Is your feature request related to a problem? Please describe.**
I would like to use sktime in the context of tree based models for time series. In general, these models need some data preparation to make them useful for time series. One data prepation step involves creating calendar dummy features representing the current day of the year, day of the month, week of the quarter etc. Another data preparation step would involve generating features representing fourier terms of different order and periodicity.

**Describe the solution you'd like**
I have been thinking about the best way to implement this, and @aiwalter  suggested to use an SeriesToSeries Transformer to generate new exogeneous features based on the index in ForecastingPipeline. Would that approach work from an architecture point of view, or do we need a separate not yet defined class "FeatureExtractor" and / or a specific IndexToSeries Generator?

We could either try to adapt solutions from GluonTS (dummies) / Prophet (fourier features) or build our own approach. Attached is some example code (not yet adapted to sktime) to outline the general goal.

```python
import pandas as pd
import numpy as np

base_seasons = [
    ["parent","child","period","dummy"],
    ["year","year",None,"year"],
    ["year","quarter",365.25/4,"quarter"],  
    ["year","month",12,"month"],
    ["year","week",365.25/7,"week_of_year"],
    ["year","day",365.25,"day_of_year"],
    ["quarter","month",12/4,"month_of_quarter"],
    ["quarter","week",365.25/(4*7),"week_of_quarter"],
    ["quarter","day",365.25/4,"day_of_quarter"],
    ["month","week",365.25/(12*7),"week_of_month"],
    ["month","day",30,"day"],
    ["week","day",7,"day_of_week"],
    ["day","hour",24,"hour"],
    ["hour","minute",60,"minute"],
    ["minute","second",60,"second"],
    ["second","millisecond",1000,"millisecond"]
]


base_seasons = pd.DataFrame(base_seasons[1:],columns=base_seasons[0])

base_seasons["fourier"] = base_seasons["child"] + '_in_' + base_seasons["parent"]
base_seasons["child"] = base_seasons["child"].astype("category").cat.reorder_categories(['year','quarter','month', 'week','day',"hour","minute","second","millisecond"])
base_seasons["rank"] = base_seasons["child"].cat.codes



def get_supported_seasons(base_frequency,base_seasons=base_seasons):
    rank = base_seasons.loc[base_seasons["child"]==base_frequency,"rank"].max()
    matches = base_seasons.loc[base_seasons["rank"]<=rank]
    if matches.shape[0] == 0:
        raise ValueError("Seasonality or Frequency not supported")
    return matches

def calendar_fourier(dti_actual,fourier_period,fourier_order,base_frequency):
    dti = pd.date_range(dti_actual.min(),dti_actual.max(),freq=base_frequency.upper()[0]) 
    
    if dti.min().to_numpy() != dti_actual.min().to_numpy():
        raise ValueError("Actual time Series does not correspond to frequencies provided by pandas Datetimeindex. This can happen when e.g. monthly data does not correspond to month end.")
    
    funcs = [np.sin,np.cos] 

    outlist = list()
    for index,item in enumerate(fourier_order):
        outlist.append((np.arange(item)+1)*1/fourier_period[index])

    inlist = np.zeros(shape=(len(dti),np.concatenate(outlist).shape[0]*len(funcs)))
    colnames = list()
    k = 0
    for item in outlist:
        for index,p in enumerate(item):
            for func in funcs:
                inlist[:,k] =func(2*np.pi*p*np.arange(len(dti)))
#                colnames.append("per"+ str(int(1/p)*int(index+1)) + "_or" + str(int(index+1))+func.__name__)
                colnames.append("per"+ str(int(1/p)) + func.__name__)
                k = k +1

    inlist = pd.DataFrame(inlist)
    inlist.columns = colnames
    inlist.set_index(dti,inplace=True)

    inlist = inlist[inlist.index.isin(dti_actual)]
    inlist = inlist.reset_index(drop=True)
    return inlist

def calendar_dummies(x,funcs):
    if funcs == "week_of_year":
        return pd.DataFrame({funcs:getattr(x,"isocalendar")()["week"].reset_index(drop="date")})
    elif funcs == "week_of_month":
        return  pd.DataFrame({funcs:(x.day - 1) // 7 + 1})
    elif funcs == "month_of_quarter":
        return pd.DataFrame({funcs:(np.floor(x.month/4)+1).astype(np.int64)})
    elif funcs == "week_of_quarter":
        year_week = getattr(x,"isocalendar")()["week"]
        def week_of_quarter(x):
            if x <= 13:
                return 1
            elif x <=26:
                return x-13
            elif x <=39:
                return x-26
            elif x <=53:
                return x-39
        year_week.apply(lambda x: week_of_quarter(x))
    elif funcs == "millisecond":
        return  pd.DataFrame({funcs:x.microsecond*1000})
    elif funcs == "day_of_quarter":
        quarter = x.quarter
        quarter_start = pd.DatetimeIndex(
        x.year.map(str) + "-" + (3 * quarter - 2).map(int).map(str) + "-01")
        values  = ((x - quarter_start) / pd.to_timedelta("1D") + 1).astype(int)
        return pd.DataFrame({funcs:values},dtype=np.float64)
    else:
        return pd.DataFrame({funcs:getattr(x,funcs)})


def calendar_other(x,funcs):
    if funcs == "proportion_total":
        values = 1+(x.view(np.int64)-x.view(np.int64).max())/(x.view(np.int64).max()-x.view(np.int64).min())
        return pd.DataFrame({funcs:values},dtype=np.float64)
    elif funcs == "proportion_total_squared":
        values = (1+(x.view(np.int64)-x.view(np.int64).max())/(x.view(np.int64).max()-x.view(np.int64).min()))^2
        return pd.DataFrame({funcs:values},dtype=np.float64)
    elif funcs == "proportion_total_cubic":
        values = (1+(x.view(np.int64)-x.view(np.int64).max())/(x.view(np.int64).max()-x.view(np.int64).min()))^3
        return pd.DataFrame({funcs:values},dtype=np.float64)
    elif funcs == "proportion_total_squared_root":
        values = (1+(x.view(np.int64)-x.view(np.int64).max())/(x.view(np.int64).max()-x.view(np.int64).min()))^(1/2)
        return pd.DataFrame({funcs:values},dtype=np.float64)
    elif funcs == "proportion_total_cubic_root":
        values = (1+(x.view(np.int64)-x.view(np.int64).max())/(x.view(np.int64).max()-x.view(np.int64).min()))^(1/3)
        return pd.DataFrame({funcs:values},dtype=np.float64)
    elif funcs == "proportion_month":
        values = x.day/x.days_in_month
        return pd.DataFrame({funcs:values},dtype=np.float64)
    elif funcs == "proportion_quarter":
        quarter = x.dt.quarter
        quarter_start = pd.DatetimeIndex(
        x.dt.year.map(str) + "-" + (3 * quarter - 2).map(int).map(str) + "-01")
        next_quarter_start = x + pd.tseries.offsets.QuarterBegin(startingMonth=1)
        quarter_length = (next_quarter_start - quarter_start).dt.days
        doq  = ((x - quarter_start) / pd.to_timedelta("1D") + 1).astype(int)
        values = doq / quarter_length
        return pd.DataFrame({funcs:values},dtype=np.float64)
```

Hints from issue discussion:
Yes that would be a great addition! 

In terms of design, the main issue is how to integrate this into the pipeline. The workflow for featurizers is slightly different compared to standard series-to-series transformers. During prediction, you first need to generate features before calling the predict method of the forecaster. 

We've had some previous discussions on this (see e.g. these [notes](https://github.com/sktime/enhancement-proposals/blob/main/steps/01_forecasting_api/forecasting_pipeline.md)). 

My preference would be to define a new class for featurizers which closely follows the forecaster API with `fit` and `predict` and extend the pipeline class to handle that.  
but why can it not be a transformer? As the calendar information has to be passed as `X`, the `ForecastingPipeline` would anyway transform `X`, so it would just add the columns?
Yes, exactly, why not?

If it's a transformer applied to `X` (not `y`), in predict it would be applied to `X` *before* predict is called, and the transformed `X` (with calendar indicator) would be fed to predict. So it seems all is fine?

My thinking was that you may also want to generate these features if you only have a target series and no exogenous data. That wouldn't be covered by the transformer. 
@mloning for this case there would be some solutions:

1. Using `ForecastingPipeline`, so the `CalendarTransformer` would also add columns if `Z is None`. So given a univariate data where user wants to add calendar data as `X`, then the user must use a `ForecastingPipeline` and we would have to accept `Z is None`. If user wants to also transform `y` then nested with with a `TransformedTargetForecaster`.

2. Other solution would be this:
```python
forecaster = TransformedTargetForecaster(...)
forecaster.fit(y, X=CalendarTransformer().fit_transform(None))
```
So we would have to accept `Z is None`. 

3. Otherwise the user would have to give a dummy `DataFrame` like this:
```python
forecaster = TransformedTargetForecaster(...)
forecaster.fit(y, X=CalendarTransformer().fit_transform(pd.DataFrame(index=y.index)))
```

Yes, the `X=None` is an unfortunate edge case here.

I would actually treat it as follows:
* change to base class: if `X` is None in `fit`, the input to `X` in `_fit` is an empty `pd.DataFrame` with the same index as `y`.
* may require some dependent changes that checks for `X` is `None` in `_fit`.
* `CalendarTransformer` only looks at the index and adds the features there.
* pipeline applied to this `X` should then work out of the box, no edge case treatment necessary
Still seems like a tricky problem.

Passing an empty `pd.DataFrame` seems unintuitive to me. In a way, empty data is also an indicator of an error so we may want to check for that at some point in the future. 

Allowing `None` in the transformer interface would be a major interface change. 

> change to base class: if X is None in fit, the input to X in _fit is an empty pd.DataFrame with the same index as y.

When we do this change, would we not have to update all forecasters that use `X` to now ignore an empty `X`? Making all forecaster having to deal with an empty `X` internally doesn't seem right to me. `None` is the expected value here for an optional input argument (for users and developers).   

To me, this seems like a perfect use case for making some progress on the more advanced pipelining, but that would involve some extra design work I think. 
or, we could simply decide that it's not worth letting this edge case stall development and lead to major design discussions, and simply add the caveat that calendar feature adder only works if you have a non-empty `X` and be done with it'?
Yes, that's also what I would prefer if no-one has the capacity to work on the pipeline, so happy to add the holiday featurizer now and think about the pipeline later. 

The simplest pipeline solution I can think of would be add a new estimator type (i.e. a featurizer) and to make distinction inside the pipeline based on estimator types.
I think also its a good idea to just ignore this edge case for now. And there is still a workaround for advanced users with the solution I mentioned:

> forecaster.fit(y, X=CalendarTransformer().fit_transform(pd.DataFrame(index=y.index)))

Commit message: DocstringFix

Patch to review:
diff --git a/sktime/transformations/series/calendardummies.py b/sktime/transformations/series/calendardummies.py
new file mode 100644
index 00000000000..d2339ba8db9
--- /dev/null
+++ b/sktime/transformations/series/calendardummies.py
@@ -0,0 +1,251 @@
+#!/usr/bin/env python3 -u
+# -*- coding: utf-8 -*-
+# copyright: sktime developers, BSD-3-Clause License (see LICENSE file)
+"""Extract calendar features from datetimeindex."""
+__author__ = ["Daniel Bartling"]
+__all__ = ["CalendarDummies"]
+
+import warnings
+
+import numpy as np
+import pandas as pd
+
+from sktime.transformations.base import _SeriesToSeriesTransformer
+from sktime.utils.validation.series import check_series
+
+base_seasons = [
+    ["parent", "child", "period", "dummy_func", "complexity"],
+    ["year", "year", None, "year", 0],
+    ["year", "quarter", 365.25 / 4, "quarter", 1],
+    ["year", "month", 12, "month", 0],
+    ["year", "week", 365.25 / 7, "week_of_year", 1],
+    ["year", "day", 365.25, "day_of_year", 1],
+    ["quarter", "month", 12 / 4, "month_of_quarter", 2],
+    ["quarter", "week", 365.25 / (4 * 7), "week_of_quarter", 2],
+    ["quarter", "day", 365.25 / 4, "day_of_quarter", 2],
+    ["month", "week", 365.25 / (12 * 7), "week_of_month", 2],
+    ["month", "day", 30, "day", 0],
+    ["week", "day", 7, "day_of_week", 0],
+    ["day", "hour", 24, "hour", 0],
+    ["hour", "minute", 60, "minute", 0],
+    ["minute", "second", 60, "second", 0],
+    ["second", "millisecond", 1000, "millisecond", 0],
+]
+
+
+date_order = [
+    "year",
+    "quarter",
+    "month",
+    "week",
+    "day",
+    "hour",
+    "minute",
+    "second",
+    "millisecond",
+]
+
+base_seasons = pd.DataFrame(base_seasons[1:], columns=base_seasons[0])
+
+base_seasons["fourier"] = base_seasons["child"] + "_in_" + base_seasons["parent"]
+base_seasons["dummy"] = base_seasons["child"] + "_of_" + base_seasons["parent"]
+base_seasons["child"] = (
+    base_seasons["child"].astype("category").cat.reorder_categories(date_order)
+)
+base_seasons["rank"] = base_seasons["child"].cat.codes
+
+
+class CalendarDummies(_SeriesToSeriesTransformer):
+    """Calendar Dummy Value Extraction for use in e.g. tree based models.
+
+    Parameters
+    ----------
+    base_frequency : str, optional (default=None)
+        Restricts selection of items to those with frequeny higher than
+        the base frequency.
+        E.g. if monthly data is provided, it does not make sense to derive
+        weekly dummies
+        Currently has to be provided by the user due to the abundance of different
+         frequencies supported by Pandas (e.g. every 4 days frequency)
+        * year
+        * quarter
+        * month
+        * week
+        * day
+        * hour
+        * minute
+        * second
+        * millisecond
+        complexity: 0,1,2, optional (default =1)
+        Specify how many calendar features you want to be returned.
+        E.g., rarely used features like week of quarter will only be returned
+        with complexity = 2.
+    manual_selection:
+        Manual selection of dummys. Notation is child of parent for precise notation.
+        Will ignore specified complexity, but will still check with warning against
+        a specified base_frequency
+        Examples:
+        * day_of_year
+        * day_of_month
+        * day_of_quarter
+        Features year_of_year as dummy (there is no supported hierarchy above year)
+
+    Example
+    -------
+    >>> from sktime.transformations.series.calendardummies import CalendarDummies
+    >>> from sktime.datasets import load_airline
+    >>> y = load_airline()
+    >>> transformer = CalendarDummies(base_frequency="month")
+    >>> y_hat = transformer.fit_transform(y)
+    """
+
+    _tags = {"fit-in-transform": True}
+
+    def __init__(self, base_frequency=None, complexity=1, manual_selection=None):
+
+        self.base_frequency = base_frequency
+
+        if (self.base_frequency is not None) & (
+            self.base_frequency not in base_seasons["child"].unique()
+        ):
+            raise ValueError(
+                "Invalid base_frequency specified, must be in: "
+                + ", ".join(base_seasons["child"].unique())
+            )
+
+        self.complexity = complexity
+
+        if self.complexity not in [0, 1, 2]:
+            raise ValueError(
+                "Invalid complexity specified,"
+                + "must be in 0,1 or 2 (0 lowest number of variables)"
+            )
+        self.manual_selection = manual_selection
+        if (self.manual_selection is not None) and (
+            not all(
+                elem in base_seasons["dummy"].unique() for elem in self.manual_selection
+            )
+        ):
+            raise ValueError(
+                "Invalid manual_selection specified, must be in: "
+                + ", ".join(base_seasons["dummy"].unique())
+            )
+
+        super(CalendarDummies, self).__init__()
+
+    def transform(self, Z, X=None):
+        """Transform data.
+
+        Returns a transformed version of Z.
+
+        Parameters
+        ----------
+        Z : pd.Series, pd.DataFrame
+
+        Returns
+        -------
+        Z : pd.Series, pd.DataFrame
+            Transformed time series(es).
+        """
+        self.check_is_fitted()
+        Z = check_series(Z)
+        Z = Z.copy()
+
+        x_df = pd.DataFrame(index=Z.index)
+        x_df["date_sequence"] = Z.index.to_timestamp().astype("datetime64[ns]")
+
+        if self.manual_selection is None:
+            if self.base_frequency is not None:
+                supported = _get_supported_calendar(self.base_frequency)
+                supported = supported[supported["complexity"] <= self.complexity]
+                calendar_dummies = supported["dummy_func"].to_list()
+            else:
+                supported = base_seasons[base_seasons["complexity"] <= self.complexity]
+                calendar_dummies = supported["dummy_func"].to_list()
+        else:
+            if self.base_frequency is not None:
+                supported = _get_supported_calendar(self.base_frequency)
+                if not all(
+                    elem in supported["dummy"] for elem in self.manual_selection
+                ):
+                    warnings.warn(
+                        "Selected dummy variable derive"
+                        + "at lower level than base frequency."
+                    )
+                calendar_dummies = supported.loc[
+                    supported["dummy"].isin(self.manual_selection), "dummy_func"
+                ]
+            else:
+                calendar_dummies = base_seasons.loc[
+                    base_seasons["dummy"].isin(self.manual_selection), "dummy_func"
+                ]
+
+        df = [_calendar_dummies(x_df, dummy) for dummy in calendar_dummies]
+        df = pd.concat(df, axis=1)
+
+        Z = pd.concat([Z, df], axis=1)
+
+        return Z
+
+
+def _calendar_dummies(x, funcs):
+    date_sequence = x["date_sequence"].dt
+    if funcs == "week_of_year":
+        # The first week of an ISO year is the first (Gregorian)
+        # calendar week of a year containing a Thursday.
+        # So it is possible that a week in the new year is still
+        # indexed starting in last year (week 52 or 53)
+        x[funcs] = date_sequence.isocalendar()["week"]
+        return x[funcs]
+    elif funcs == "week_of_month":
+        x[funcs] = (date_sequence.day - 1) // 7 + 1
+        return x[funcs]
+    elif funcs == "month_of_quarter":
+        x[funcs] = (np.floor(date_sequence.month / 4) + 1).astype(np.int64)
+        return x[funcs]
+    elif funcs == "week_of_quarter":
+        col_names = x.columns
+        x_columns = col_names.intersection(["year", "quarter", "week"]).to_list()
+        x_columns.append("date_sequence")
+        df = x.copy(deep=True)
+        df = df[x_columns]
+        if "year" not in x_columns:
+            df["year"] = df["date_sequence"].dt.year
+        if "quarter" not in x_columns:
+            df["quarter"] = df["date_sequence"].dt.quarter
+        if "week" not in x_columns:
+            df["week"] = df["date_sequence"].dt.isocalendar()["week"]
+        df["qdate"] = (
+            df["date_sequence"] + pd.tseries.offsets.DateOffset(days=1)
+        ) - pd.tseries.offsets.QuarterBegin(startingMonth=1)
+        df["qweek"] = df["qdate"].dt.isocalendar()["week"]
+        df.loc[(df["quarter"] == 1) & (df["week"] < 52), "qweek"] = 0
+        df["week_of_quarter"] = df["week"] - df["qweek"] + 1
+        return df["week_of_quarter"]
+    elif funcs == "millisecond":
+        x[funcs] = date_sequence.microsecond * 1000
+        return x[funcs]
+    elif funcs == "day_of_quarter":
+        quarter = date_sequence.quarter
+        quarter_start = pd.DatetimeIndex(
+            date_sequence.year.map(str)
+            + "-"
+            + (3 * quarter - 2).map(int).map(str)
+            + "-01"
+        )
+        values = (
+            (x["date_sequence"] - quarter_start) / pd.to_timedelta("1D") + 1
+        ).astype(int)
+        x[funcs] = values
+        return x[funcs]
+    else:
+        x[funcs] = getattr(date_sequence, funcs)
+        return x[funcs]
+
+
+def _get_supported_calendar(base_frequency, base_seasons=base_seasons):
+    rank = base_seasons.loc[base_seasons["child"] == base_frequency, "rank"].max()
+    matches = base_seasons.loc[base_seasons["rank"] <= rank]
+    if matches.shape[0] == 0:
+        raise ValueError("Seasonality or Frequency not supported")
+    return matches
diff --git a/sktime/transformations/series/tests/tests_calendardummies.py b/sktime/transformations/series/tests/tests_calendardummies.py
new file mode 100644
index 00000000000..50339f4bd98
--- /dev/null
+++ b/sktime/transformations/series/tests/tests_calendardummies.py
@@ -0,0 +1,193 @@
+#!/usr/bin/env python3 -u
+# -*- coding: utf-8 -*-
+# copyright: sktime developers, BSD-3-Clause License (see LICENSE file).
+"""Unit tests of CalendarDummies functionality."""
+
+# %%
+import pandas as pd
+import pytest
+
+from sktime.datasets import load_airline, load_longley
+from sktime.forecasting.base import ForecastingHorizon
+from sktime.forecasting.compose import ForecastingPipeline
+from sktime.forecasting.model_selection import temporal_train_test_split
+from sktime.forecasting.naive import NaiveForecaster
+from sktime.transformations.series.calendardummies import CalendarDummies
+
+y, X = load_longley()
+y_train, y_test, X_train, X_test = temporal_train_test_split(y, X)
+
+fh = ForecastingHorizon(X_test.index, is_relative=False)
+pipe = ForecastingPipeline(
+    steps=[
+        (
+            "CalendarDummies",
+            CalendarDummies(
+                base_frequency="week", complexity=2, manual_selection=["week_of_year"]
+            ),
+        ),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+first_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+pipe = ForecastingPipeline(
+    steps=[
+        ("CalendarDummies", CalendarDummies(base_frequency="week", complexity=0)),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+second_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+pipe = ForecastingPipeline(
+    steps=[
+        ("CalendarDummies", CalendarDummies(base_frequency="month", complexity=2)),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+third_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+
+pipe = ForecastingPipeline(
+    steps=[
+        (
+            "CalendarDummies",
+            CalendarDummies(
+                base_frequency="month",
+                complexity=2,
+                manual_selection=["year_of_year", "second_of_minute"],
+            ),
+        ),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+fourth_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+
+pipe = ForecastingPipeline(
+    steps=[
+        (
+            "CalendarDummies",
+            CalendarDummies(manual_selection=["year_of_year", "second_of_minute"]),
+        ),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+fifth_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+# %%
+y, X = load_airline(), pd.DataFrame(index=load_airline().index)
+y_train, y_test, X_train, X_test = temporal_train_test_split(y, X)
+
+pipe = ForecastingPipeline(
+    steps=[
+        (
+            "CalendarDummies",
+            CalendarDummies(manual_selection=["year_of_year", "second_of_minute"]),
+        ),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+sixth_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+test = load_airline()
+test.index = test.index.view("int64")
+y, X = test, pd.DataFrame(index=test.index)
+
+pipe = ForecastingPipeline(
+    steps=[
+        (
+            "CalendarDummies",
+            CalendarDummies(manual_selection=["year_of_year", "second_of_minute"]),
+        ),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+sixth_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+y, X = load_airline(), pd.DataFrame(index=load_airline().index)
+y_train, y_test, X_train, X_test = temporal_train_test_split(y, X)
+
+pipe = ForecastingPipeline(
+    steps=[
+        (
+            "CalendarDummies",
+            CalendarDummies(manual_selection=["year_of_year", "second_of_minute"]),
+        ),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+seventh_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+
+
+@pytest.mark.parametrize(
+    "test_input,expected",
+    [
+        (
+            first_test,
+            [
+                "GNPDEFL",
+                "GNP",
+                "UNEMP",
+                "ARMED",
+                "POP",
+                "year",
+                "quarter",
+                "month",
+                "week_of_year",
+                "month_of_quarter",
+                "week_of_quarter",
+                "week_of_month",
+            ],
+        ),
+        (second_test, ["GNPDEFL", "GNP", "UNEMP", "ARMED", "POP", "year", "month"]),
+        (
+            third_test,
+            [
+                "GNPDEFL",
+                "GNP",
+                "UNEMP",
+                "ARMED",
+                "POP",
+                "year",
+                "quarter",
+                "month",
+                "month_of_quarter",
+            ],
+        ),
+        (fourth_test, ["GNPDEFL", "GNP", "UNEMP", "ARMED", "POP", "year"]),
+        (fifth_test, ["GNPDEFL", "GNP", "UNEMP", "ARMED", "POP", "year", "second"]),
+        (sixth_test, ["year", "second"]),
+        (seventh_test, ["year", "second"]),
+    ],
+)
+def test_eval(test_input, expected):
+    """Tests assertions."""
+    assert len(test_input) == len(expected)
+    assert all([a == b for a, b in zip(test_input, expected)])
+
+
+# %%


Code review comments:
diff_hunk: @@ -0,0 +1,251 @@
+#!/usr/bin/env python3 -u
+# -*- coding: utf-8 -*-
+# copyright: sktime developers, BSD-3-Clause License (see LICENSE file)
+"""Extract calendar features from datetimeindex."""
+__author__ = ["Daniel Bartling"]
+__all__ = ["CalendarDummies"]
+
+import warnings
+
+import numpy as np
+import pandas as pd
+
+from sktime.transformations.base import _SeriesToSeriesTransformer
+from sktime.utils.validation.series import check_series
+
+base_seasons = [
+    ["parent", "child", "period", "dummy_func", "complexity"],
+    ["year", "year", None, "year", 0],
+    ["year", "quarter", 365.25 / 4, "quarter", 1],
+    ["year", "month", 12, "month", 0],
+    ["year", "week", 365.25 / 7, "week_of_year", 1],
+    ["year", "day", 365.25, "day_of_year", 1],
+    ["quarter", "month", 12 / 4, "month_of_quarter", 2],
+    ["quarter", "week", 365.25 / (4 * 7), "week_of_quarter", 2],
+    ["quarter", "day", 365.25 / 4, "day_of_quarter", 2],
+    ["month", "week", 365.25 / (12 * 7), "week_of_month", 2],
+    ["month", "day", 30, "day", 0],
+    ["week", "day", 7, "day_of_week", 0],
+    ["day", "hour", 24, "hour", 0],
+    ["hour", "minute", 60, "minute", 0],
+    ["minute", "second", 60, "second", 0],
+    ["second", "millisecond", 1000, "millisecond", 0],
+]
+
+
+date_order = [
+    "year",
+    "quarter",
+    "month",
+    "week",
+    "day",
+    "hour",
+    "minute",
+    "second",
+    "millisecond",
+]
+
+base_seasons = pd.DataFrame(base_seasons[1:], columns=base_seasons[0])
+
+base_seasons["fourier"] = base_seasons["child"] + "_in_" + base_seasons["parent"]
+base_seasons["dummy"] = base_seasons["child"] + "_of_" + base_seasons["parent"]
+base_seasons["child"] = (
+    base_seasons["child"].astype("category").cat.reorder_categories(date_order)
+)
+base_seasons["rank"] = base_seasons["child"].cat.codes
+
+
+class CalendarDummies(_SeriesToSeriesTransformer):
+    """Calendar Dummy Value Extraction for use in e.g. tree based models.
+
+    Parameters
+    ----------
+    base_frequency : str, optional (default=None)
+        Restricts selection of items to those with frequeny higher than
+        the base frequency.
+        E.g. if monthly data is provided, it does not make sense to derive
+        weekly dummies
+        Currently has to be provided by the user due to the abundance of different
+         frequencies supported by Pandas (e.g. every 4 days frequency)
+        * year
+        * quarter
+        * month
+        * week
+        * day
+        * hour
+        * minute
+        * second
+        * millisecond
+        complexity: 0,1,2, optional (default =1)
+        Specify how many calendar features you want to be returned.
+        E.g., rarely used features like week of quarter will only be returned
+        with complexity = 2.
+    manual_selection:
+        Manual selection of dummys. Notation is child of parent for precise notation.
+        Will ignore specified complexity, but will still check with warning against
+        a specified base_frequency
+        Examples:
+        * day_of_year
+        * day_of_month
+        * day_of_quarter
+        Features year_of_year as dummy (there is no supported hierarchy above year)
+
+    Example
+    -------
+    >>> from sktime.transformations.series.calendardummies import CalendarDummies
+    >>> from sktime.datasets import load_airline
+    >>> y = load_airline()
+    >>> transformer = CalendarDummies(base_frequency="month")
+    >>> y_hat = transformer.fit_transform(y)
+    """
+
+    _tags = {"fit-in-transform": True}
path: sktime/transformations/series/calendardummies.py
text: @user1:
we need to add some more tags here, you can find them here: https://github.com/alan-turing-institute/sktime/blob/f869fa42b130b3525f702caab3bb710358b8e106/sktime/registry/_tags.py


@user1:
we need `"skip-inverse-transform": True` and `"univariate-only": False` 

@author:
Integrated

diff_hunk: @@ -0,0 +1,193 @@
+#!/usr/bin/env python3 -u
+# -*- coding: utf-8 -*-
+# copyright: sktime developers, BSD-3-Clause License (see LICENSE file).
+"""Unit tests of CalendarDummies functionality."""
+
+# %%
+import pandas as pd
+import pytest
+
+from sktime.datasets import load_airline, load_longley
+from sktime.forecasting.base import ForecastingHorizon
+from sktime.forecasting.compose import ForecastingPipeline
+from sktime.forecasting.model_selection import temporal_train_test_split
+from sktime.forecasting.naive import NaiveForecaster
+from sktime.transformations.series.calendardummies import CalendarDummies
+
+y, X = load_longley()
+y_train, y_test, X_train, X_test = temporal_train_test_split(y, X)
+
+fh = ForecastingHorizon(X_test.index, is_relative=False)
+pipe = ForecastingPipeline(
+    steps=[
+        (
+            "CalendarDummies",
+            CalendarDummies(
+                base_frequency="week", complexity=2, manual_selection=["week_of_year"]
+            ),
+        ),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+first_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+pipe = ForecastingPipeline(
+    steps=[
+        ("CalendarDummies", CalendarDummies(base_frequency="week", complexity=0)),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+second_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+pipe = ForecastingPipeline(
+    steps=[
+        ("CalendarDummies", CalendarDummies(base_frequency="month", complexity=2)),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+third_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+
+pipe = ForecastingPipeline(
+    steps=[
+        (
+            "CalendarDummies",
+            CalendarDummies(
+                base_frequency="month",
+                complexity=2,
+                manual_selection=["year_of_year", "second_of_minute"],
+            ),
+        ),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+fourth_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+
+pipe = ForecastingPipeline(
+    steps=[
+        (
+            "CalendarDummies",
+            CalendarDummies(manual_selection=["year_of_year", "second_of_minute"]),
+        ),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+fifth_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+# %%
+y, X = load_airline(), pd.DataFrame(index=load_airline().index)
path: sktime/transformations/series/tests/tests_calendardummies.py
text: @user1:
why this? can we just use `load_longley`?

@author:
I want to test on  a multivariate and univariate data set

@user1:
yes then use `y, X = load_longley()`

@author:
But longley is multivariate not univariate? I use longley for some test, and airline for other, to make sure it works for both multivariate and univariate

@user1:
`load_longley()` returns univariate `y` and multivariate `X`. I you need multivariate `y`, just do `_, y = load_longley()`

@author:
ok I modified airline to become univariate without index construction

diff_hunk: @@ -0,0 +1,193 @@
+#!/usr/bin/env python3 -u
+# -*- coding: utf-8 -*-
+# copyright: sktime developers, BSD-3-Clause License (see LICENSE file).
+"""Unit tests of CalendarDummies functionality."""
+
+# %%
+import pandas as pd
+import pytest
+
+from sktime.datasets import load_airline, load_longley
+from sktime.forecasting.base import ForecastingHorizon
+from sktime.forecasting.compose import ForecastingPipeline
+from sktime.forecasting.model_selection import temporal_train_test_split
+from sktime.forecasting.naive import NaiveForecaster
+from sktime.transformations.series.calendardummies import CalendarDummies
+
+y, X = load_longley()
+y_train, y_test, X_train, X_test = temporal_train_test_split(y, X)
+
+fh = ForecastingHorizon(X_test.index, is_relative=False)
+pipe = ForecastingPipeline(
path: sktime/transformations/series/tests/tests_calendardummies.py
text: @user1:
Do we actually need to have a `ForecastingPipeline` here in tests? In the end we just test the transformer?

@author:
No good point, removed it in latest version

diff_hunk: @@ -0,0 +1,193 @@
+#!/usr/bin/env python3 -u
+# -*- coding: utf-8 -*-
+# copyright: sktime developers, BSD-3-Clause License (see LICENSE file).
+"""Unit tests of CalendarDummies functionality."""
+
+# %%
+import pandas as pd
+import pytest
+
+from sktime.datasets import load_airline, load_longley
+from sktime.forecasting.base import ForecastingHorizon
+from sktime.forecasting.compose import ForecastingPipeline
+from sktime.forecasting.model_selection import temporal_train_test_split
+from sktime.forecasting.naive import NaiveForecaster
+from sktime.transformations.series.calendardummies import CalendarDummies
+
+y, X = load_longley()
+y_train, y_test, X_train, X_test = temporal_train_test_split(y, X)
+
+fh = ForecastingHorizon(X_test.index, is_relative=False)
+pipe = ForecastingPipeline(
+    steps=[
+        (
+            "CalendarDummies",
+            CalendarDummies(
+                base_frequency="week", complexity=2, manual_selection=["week_of_year"]
+            ),
+        ),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+first_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+pipe = ForecastingPipeline(
+    steps=[
+        ("CalendarDummies", CalendarDummies(base_frequency="week", complexity=0)),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+second_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+pipe = ForecastingPipeline(
+    steps=[
+        ("CalendarDummies", CalendarDummies(base_frequency="month", complexity=2)),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+third_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
+
+pipe = ForecastingPipeline(
+    steps=[
+        (
+            "CalendarDummies",
+            CalendarDummies(
+                base_frequency="month",
+                complexity=2,
+                manual_selection=["year_of_year", "second_of_minute"],
+            ),
+        ),
+        ("forecaster", NaiveForecaster(strategy="drift")),
+    ]
+)
+
+pipe.steps[0][1].fit(Z=X_train)
+fourth_test = pipe.steps[0][1].transform(Z=X_train).columns.to_list()
+
+# %%
path: sktime/transformations/series/tests/tests_calendardummies.py
text: @user1:
pls remove this kind of lines