Repository: sktime/sktime

Issue:
[ENH] `tsfresh` transformers - improved signature, custom features, docstring
update by @fkiraly:

`TSFreshFeatureExtractor` is not in a good state, we need:

* a clean and clear docstring, which also describes parameters, and explains pre-defined and user specified parameter sets.
* a parameter that allows the user to specify the features they would like to extract (this may already exist)
* possibly some changes in what the parameters are to make it more user friendly (may need deprecation)
* `get_test_params` with at least two test cases, one with prespecified parameter set and one with a custom specified parameter set.

---

**Is your feature request related to a problem? Please describe.**

Hi, I'm not sure if my issue is related to something not implemented or if I just don't know how to perfom it.
Basically, I would like to use the extract_features() method of tsfresh just to extract a subset of features defined in a dictionary, which would be different than the 'minimal','efficient' or 'comprehensive' option. 
If possible still using default parameters for this subset of features: 

```
from sktime.transformations.panel.tsfresh import TSFreshFeatureExtractor

features_to_calc = [
    'absolute_sum_of_changes',
    'absolute_maximum',
    'agg_linear_trend',
    'large_standard_deviation',
    'longest_strike_above_mean',
    'longest_strike_below_mean',
    'mean_abs_change',
    'mean_change',
    'quantile',
    'variance_larger_than_standard_deviation',
    'variance',
]

features_dict = dict.fromkeys(features_to_calc, None)
```

If I try the following, it extracts every features of the default or other settings (which is obviously very long except for minimal setting), without considering my dictionary (name of features as keys, None as values)

```
from sktime.transformations.panel.tsfresh import TSFreshFeatureExtractor
tsfresh = TSFreshFeatureExtractor(kind_to_fc_parameters=features_dict)

X_transform = tsfresh.fit_transform(panel_data)
```

Am I missing something or is it just not implemented yet for consistency with other transformers?

**Describe the solution you'd like**

Would it be possible to extend tsfresh extract_features() method to panel dataset in sktime, for instance using the following:

```
from sktime.transformations.panel.tsfresh import TSFreshFeatureExtractor
tsfresh = TSFreshFeatureExtractor(kind_to_fc_parameters=features_dict)

X_transform = tsfresh.extract_features(panel_data)
```

**Describe alternatives you've considered**
I referred to  https://github.com/blue-yonder/tsfresh/blob/main/notebooks/examples/03%20Feature%20Extraction%20Settings.ipynb
but only the not so convenient tsfresh dataformat are supported, so maybe instead I should transform my data to fit in one of the tsfresh format? If this case is the easiest, I could use a few lines to help since it is not clear how could i get there with the page https://www.sktime.org/en/stable/examples/AA_datatypes_and_datasets.html

**Additional context**
Relative to that alternative solution, completing the datasets types notebook referred previously with a canva, to convert from one or a few tsfresh format to one or a few sktime format would be ideal (only for a few ultivariate options which are more convenient from one package to the other...)

Thanks very much in advance for your help, and for your work,

Julien


Discussion:
ok, I think there are two issues here:
* support for user specified feature set
* support for panel data (features per instance)

The second should be out-of-the-box, if you pass a panel data container to the transformer, it applies the transformation to each instance separately.

From a glance at the code, the first is indeed not implemented, and the transformer itself is badly documented, this needs to be improved.
I turned this issue into an "improve the tsfresh feature extractor" one since imo we need to do that. Hope that's ok.
I looked into it more, the parameter passing logic resides in `_get_extraction_params`, by that it appears that the `tsfresh` default of `default_fc_parameters=None` is overridden by `"efficient"`, perhaps that is the root cause?

Either way, having a docstring with examples, and also test parameters in `get_test_params` would be great.

Resolving patch:
diff --git a/sktime/transformations/panel/tsfresh.py b/sktime/transformations/panel/tsfresh.py
--- a/sktime/transformations/panel/tsfresh.py
+++ b/sktime/transformations/panel/tsfresh.py
@@ -33,7 +33,7 @@ class _TSFreshFeatureExtractor(BaseTransformer):
 
     def __init__(
         self,
-        default_fc_parameters="efficient",
+        default_fc_parameters=None,
         kind_to_fc_parameters=None,
         chunksize=None,
         n_jobs=1,
@@ -83,12 +83,12 @@ def _get_extraction_params(self):
             ComprehensiveFCParameters,
             EfficientFCParameters,
             MinimalFCParameters,
+            from_columns,
         )
         from tsfresh.utilities.dataframe_functions import impute
 
         # Set defaults from tsfresh
         extraction_params = {
-            "kind_to_fc_parameters": self.kind_to_fc_parameters,
             "n_jobs": N_PROCESSES,
             "chunksize": CHUNKSIZE,
             "show_warnings": SHOW_WARNINGS,
@@ -123,19 +123,120 @@ def _get_extraction_params(self):
                 )
             else:
                 fc_parameters = fc_param_lookup[self.default_fc_parameters]
+        # TODO: remove elif in v0.15.0
+        # this will pass None to tsfresh, and hence default to "comprehensive"
+        elif self.default_fc_parameters is None:
+            fc_parameters = fc_param_lookup["efficient"]
+            warn(
+                "Passing None to default_fc_parameters currently defaults "
+                "to 'efficient', this behaviour has been is deprecated. From 0.15.0, "
+                "this will change to passing default_fc_parameters directly to tsfresh,"
+                " which in turn defaults to 'comprehensive'.",
+                DeprecationWarning,
+            )
         else:
             fc_parameters = self.default_fc_parameters
         extraction_params["default_fc_parameters"] = fc_parameters
 
+        # creates mapping from kind names to fc_parameter objects
+        if self.kind_to_fc_parameters is not None:
+            self.kind_to_fc_parameters_ = from_columns(self.kind_to_fc_parameters)
+        else:
+            self.kind_to_fc_parameters_ = self.kind_to_fc_parameters
+        extraction_params["kind_to_fc_parameters"] = self.kind_to_fc_parameters_
         return extraction_params
 
 
+# todo 0.15.0: change default_fc_parameters docstring to "comprehensive" default"
 class TSFreshFeatureExtractor(_TSFreshFeatureExtractor):
     """Transformer for extracting time series features.
 
+    Parameters
+    ----------
+    default_fc_parameters : string, FCParameters object or None,
+        default=None = "efficient"
+        Specifies pre-defined feature sets to be extracted
+        If string, should be in ["minimal", "efficient", "comprehensive"]
+        See [3] for more details.
+    kind_to_fc_parameters : list or None, default=None
+        containing strings specifying selected features to be extracted.
+        The naming convention from tsfresh applies, i.e. the strings
+        should be structured as:
+        {time_series_name}__{feature_name}__{param name 1}_
+        {param value 1}__[..]__{param name k}_{param value k}.
+        See [2] for more details and [4] for viable options.
+        Either default_fc_parameters or kind_to_fc_parameters
+        should be passed. If both are passed, only features specified
+        in kind_to_fc_parameters are extracted. If neither
+        is passed, it calculates the "comprehensive"
+        feature set.
+    n_jobs : int, default=1
+        The number of processes to use for parallelization.
+        If zero, no parallelization is used.
+    chunksize : None or int, default=None
+        The size of one chunk that is submitted to the worker
+        process for the parallelisation.  Where one chunk is defined as a
+        singular time series for one id and one kind. If you set the chunksize
+        to 10, then it means that one task is to calculate all features for 10
+        time series.  If it is set it to None, depending on distributor,
+        heuristics are used to find the optimal chunksize. If you get out of
+        memory exceptions, you can try it with the dask distributor and a
+        smaller chunksize.
+    show_warnings : bool, default=True
+        Show warnings during the feature extraction
+         (needed for debugging of calculators).
+    disable_progressbar : bool, default=False
+        Do not show a progressbar while doing the calculation.
+    impute_function : None or Callable, default=None
+        None, if no imputing should happen or the function to call for
+        imputing the result dataframe. Imputing will never happen on the input data.
+    profiling : bool, default=None
+        Turn on profiling during feature extraction
+    profiling_sorting : basestring, default=None
+        How to sort the profiling results (see the documentation
+        of the profiling package for more information)
+    profiling_filename : basestring, default=None
+        Where to save the profiling results.
+    distributor : distributor class, default=None
+        Advanced parameter: set this to a class name that you want to use as a
+        distributor. See the utilities/distribution.py for more information.
+        Leave to None, if you want TSFresh to choose the best distributor.
+
+    Returns
+    -------
+    DataFrame containing extracted features
+
     References
     ----------
     .. [1]  https://github.com/blue-yonder/tsfresh
+    .. [2]  https://tsfresh.readthedocs.io/en/v0.1.2/text/feature_naming.html
+    .. [3]  https://tsfresh.readthedocs.io/en/latest/text/
+            feature_extraction_settings.html
+    .. [4]  https://tsfresh.readthedocs.io/en/latest/api/tsfresh.feature_extraction.html
+            #module-tsfresh.feature_extraction.feature_calculators
+
+    Examples
+    --------
+    >>> from sklearn.model_selection import train_test_split
+    >>> from sktime.datasets import load_arrow_head
+    >>> from sktime.transformations.panel.tsfresh import (
+    ... TSFreshFeatureExtractor
+    ... )
+    >>> X, y = load_arrow_head(return_X_y=True)
+    >>> X_train, X_test, y_train, y_test = train_test_split(X, y)
+    >>> ts_eff = TSFreshFeatureExtractor(
+    ...     default_fc_parameters="efficient", disable_progressbar=True
+    ... ) # doctest: +SKIP
+    >>> X_transform1 = ts_eff.fit_transform(X_train) # doctest: +SKIP
+    >>> features_to_calc = [
+    ...     "dim_0__quantile__q_0.6",
+    ...     "dim_0__longest_strike_above_mean",
+    ...     "dim_0__variance",
+    ... ]
+    >>> ts_custom = TSFreshFeatureExtractor(
+    ...     kind_to_fc_parameters=features_to_calc, disable_progressbar=True
+    ... ) # doctest: +SKIP
+    >>> X_transform2 = ts_custom.fit_transform(X_train) # doctest: +SKIP
     """
 
     _tags = {
@@ -210,7 +311,24 @@ def get_test_params(cls, parameter_set="default"):
             `MyClass(**params)` or `MyClass(**params[i])` creates a valid test instance.
             `create_test_instance` uses the first (or only) dictionary in `params`
         """
-        return {"disable_progressbar": True, "show_warnings": False}
+        features_to_calc = [
+            "dim_0__quantile__q_0.6",
+            "dim_0__longest_strike_above_mean",
+            "dim_0__variance",
+        ]
+
+        return [
+            {
+                "disable_progressbar": True,
+                "show_warnings": False,
+                "default_fc_parameters": "efficient",
+            },
+            {
+                "disable_progressbar": True,
+                "show_warnings": False,
+                "kind_to_fc_parameters": features_to_calc,
+            },
+        ]
 
 
 class TSFreshRelevantFeatureExtractor(_TSFreshFeatureExtractor):


Test patch:
diff --git a/sktime/transformations/panel/tests/test_tsfresh.py b/sktime/transformations/panel/tests/test_tsfresh.py
--- a/sktime/transformations/panel/tests/test_tsfresh.py
+++ b/sktime/transformations/panel/tests/test_tsfresh.py
@@ -4,7 +4,9 @@
 
 import numpy as np
 import pytest
+from sklearn.model_selection import train_test_split
 
+from sktime.datasets import load_arrow_head
 from sktime.datatypes import convert
 from sktime.transformations.panel.tsfresh import TSFreshFeatureExtractor
 from sktime.utils._testing.panel import make_classification_problem
@@ -30,3 +32,46 @@ def test_tsfresh_extractor(default_fc_parameters):
     expected = converted.mean(axis=1).values
     assert expected[0] == X.iloc[0, 0].mean()
     np.testing.assert_allclose(actual, expected)
+
+
+@pytest.mark.skipif(
+    not _check_soft_dependencies("tsfresh", severity="none"),
+    reason="skip test if required soft dependency tsfresh not available",
+)
+def test_docs_tsfresh_extractor():
+    """Test whether doc example runs through."""
+    X, y = load_arrow_head(return_X_y=True)
+    X_train, X_test, y_train, y_test = train_test_split(X, y)
+    ts_eff = TSFreshFeatureExtractor(
+        default_fc_parameters="efficient", disable_progressbar=True
+    )
+    ts_eff.fit_transform(X_train)
+    features_to_calc = [
+        "dim_0__quantile__q_0.6",
+        "dim_0__longest_strike_above_mean",
+        "dim_0__variance",
+    ]
+    ts_custom = TSFreshFeatureExtractor(
+        kind_to_fc_parameters=features_to_calc, disable_progressbar=True
+    )
+    ts_custom.fit_transform(X_train)
+
+
+@pytest.mark.skipif(
+    not _check_soft_dependencies("tsfresh", severity="none"),
+    reason="skip test if required soft dependency tsfresh not available",
+)
+def test_kind_tsfresh_extractor():
+    """Test extractor returns an array of expected num of cols."""
+    X, y = load_arrow_head(return_X_y=True)
+    X_train, X_test, y_train, y_test = train_test_split(X, y)
+    features_to_calc = [
+        "dim_0__quantile__q_0.6",
+        "dim_0__longest_strike_above_mean",
+        "dim_0__variance",
+    ]
+    ts_custom = TSFreshFeatureExtractor(
+        kind_to_fc_parameters=features_to_calc, disable_progressbar=True
+    )
+    Xts_custom = ts_custom.fit_transform(X_train)
+    assert Xts_custom.shape[1] == len(features_to_calc)