Repository: sktime/sktime

Issue:
[BUG] TypeError for ComposableTimeSeriesForestClassifier
I was trying to follow the tutorial for "Univariate time series classification with sktime" but ran into an error with the time series forest classifier `ComposableTimeSeriesForestClassifier`. Copied the code exactly, everything was fine for a single time series tree. 

Code: 
```
tsf = ComposableTimeSeriesForestClassifier(
    estimator=time_series_tree,
    n_estimators=100,
    criterion="entropy",
    bootstrap=True,
    oob_score=True,
    random_state=1,
    n_jobs=-1,
)
```

Error: 
```
TypeError                                 Traceback (most recent call last)
<ipython-input-25-a9421957ffb2> in <module>()
      7     oob_score=True,
      8     random_state=1,
----> 9     n_jobs=-1,
     10 )

TypeError: Can't instantiate abstract class ComposableTimeSeriesForestClassifier with abstract methods _set_oob_score_and_attributes
```

(Google Colab, no GPU/TPU)



Discussion:
hi,, thanks for the spot, @mloning one for you I think? 
I have looked at this, and think this class should be removed. It does not conform to docstrings and requires updating. However,  TSF is a very obscure, never used algorithm that is way behind SOTA. The functionality it encapsulates is available through the concrete implementation. I think we remove this composable classifier. Its easier than updating it and I just don’t see what value it serves. Thoughts @mloning ?
I am having the same issue - I raised a separate issue #1546 but then found this one which is the same one I am having. I am going to take a look into the issue and see if it can be easily resolved.

**My code**

```python
from sktime.transformations.panel.summarize import RandomIntervalFeatureExtractor

steps = [
    (
        "extract",
        RandomIntervalFeatureExtractor(n_intervals="sqrt", features=[np.mean, np.std, _slope])
    ),
    ("clf", DecisionTreeClassifier())
]

time_series_tree = Pipeline(steps)

# Fit and evaluate single time series tree
time_series_tree.fit(x_train, y_train)
time_series_tree.score(x_test, `y_test)`

tsf = ComposableTimeSeriesForestClassifier( # This is where the TypeError is thrown
    estimator=time_series_tree,
    n_estimators=100,
    criterion="entropy",
    bootstrap=True,
    oob_score=True,
    random_state=1,
    n_jobs=-1
)

# Fit and obtain oob score
tsf.fit(x_train, y_train)

if tsf.oob_score:
    print(tsf.oob_score_)
```

**Expected behavior**
ComposableTimeSeriesForestClassifier should be instantiable
OR 
Tutorial should be changed to show correct methodology for using time series forests.

**Additional context**
I believe the issue might be related to __init__ being marked as an abstract method [here.](1ec596b9d5464d21ffb0236730afded23/sktime/series_as_features/base/estimators/_ensemble.py#L88)

**Versions**
<details>
System:
    python: 3.8.3 (tags/v3.8.3:6f8c832, May 13 2020, 22:37:02) [MSC v.1924 64 bit (AMD64)]
executable: c:\users\olive\onedrive - university of east anglia\year 3\computing project (cmp-6013y)\datasets\venv\scripts\python.exe
   machine: Windows-10-10.0.22000-SP0

Python dependencies:
          pip: 21.1.2
   setuptools: 57.0.0
      sklearn: 1.0
       sktime: 0.8.0
  statsmodels: 0.13.0
        numpy: 1.19.5
        scipy: 1.7.1
       Cython: 0.29.24
       pandas: 1.3.4
   matplotlib: 3.4.3
       joblib: 1.1.0
        numba: 0.54.1
     pmdarima: 1.8.3
      tsfresh: 0.18.0
</details>

@OliverMatthews you could be right, have a go removing it and see if it works, would be a good first issue.
> @OliverMatthews you could be right, have a go removing it and see if it works, would be a good first issue.

Tried removing the abstract tag, didn't work but I think I've identified the issue. Up the chain of abstracted classes is an abstract method called _set_oob_score_and_attributes in sklearn/ensemble/_forest.py that is not implemented in the sktime classes that implement it.
I made some changes so that the ComposableTimeSeriesForestClassifier class is now instantiable, however it raises a ValueError when trying to use the fit method:
[Link to gist with error information (linked as a gist, as the printout is quite long)](https://gist.github.com/OliverMatthews/2ba6145cb1396fb584ada2384fe1e077#file-error_info)

It seems the value "entropy" is not valid for the attribute 'criterion', even though this is currently the default value for that attribute when instantiating ComposableTimeSeriesForestClassifier objects.

Running print(tsf.estimator.get_params().keys()) gives the following:

```
`dict_keys(['memory', 'steps', 'verbose', 'extract', 'clf', 'extract__features', 'extract__max_length', 'extract__min_length', 'extract__n_intervals', 'extract__random_state', 'clf__ccp_alpha', 'clf__class_weight', 'clf__criterion', 'clf__max_depth', 'clf__max_features', 'clf__max_leaf_nodes', 'clf__min_impurity_decrease', 'clf__min_samples_leaf', 'clf__min_samples_split', 'clf__min_weight_fraction_leaf', 'clf__random_state', 'clf__splitter'])`
```


Resolving patch:
diff --git a/sktime/classification/base.py b/sktime/classification/base.py
--- a/sktime/classification/base.py
+++ b/sktime/classification/base.py
@@ -63,7 +63,7 @@ def fit(self, X, y):
         Parameters
         ----------
         X : 2D np.array (univariate, equal length series) of shape = [n_instances,
-        series_length]
+            series_length]
             or 3D np.array (any number of dimensions, equal length series) of shape =
             [n_instances,n_dimensions,series_length]
             or pd.DataFrame with each column a dimension, each cell a pd.Series (any
diff --git a/sktime/classification/compose/_ensemble.py b/sktime/classification/compose/_ensemble.py
--- a/sktime/classification/compose/_ensemble.py
+++ b/sktime/classification/compose/_ensemble.py
@@ -1,30 +1,28 @@
 # -*- coding: utf-8 -*-
-"""
-Configurable time series ensembles
-"""
+"""Configurable time series ensembles."""
 __author__ = ["Markus Löning", "Ayushmaan Seth"]
 __all__ = ["ComposableTimeSeriesForestClassifier"]
 
-from warnings import warn
-import numpy as np
 import numbers
-from joblib import Parallel
-from joblib import delayed
+from warnings import warn
 
+import numpy as np
+from joblib import Parallel, delayed
 from sklearn.ensemble._base import _partition_estimators
+from sklearn.ensemble._forest import (
+    _generate_unsampled_indices,
+    _get_n_samples_bootstrap,
+)
+from sklearn.pipeline import Pipeline
 from sklearn.tree import DecisionTreeClassifier
 from sklearn.utils import compute_sample_weight
 from sklearn.utils.multiclass import check_classification_targets
-from sklearn.pipeline import Pipeline
-from sklearn.ensemble._forest import _generate_unsampled_indices
-from sklearn.ensemble._forest import _get_n_samples_bootstrap
-from sktime.transformations.panel.summarize import (
-    RandomIntervalFeatureExtractor,
-)
-from sktime.utils.slope_and_trend import _slope
-from sktime.utils.validation.panel import check_X, check_X_y
+
 from sktime.classification.base import BaseClassifier
 from sktime.series_as_features.base.estimators._ensemble import BaseTimeSeriesForest
+from sktime.transformations.panel.summarize import RandomIntervalFeatureExtractor
+from sktime.utils.slope_and_trend import _slope
+from sktime.utils.validation.panel import check_X, check_X_y
 
 
 class ComposableTimeSeriesForestClassifier(BaseTimeSeriesForest, BaseClassifier):
@@ -58,10 +56,6 @@ class ComposableTimeSeriesForestClassifier(BaseTimeSeriesForest, BaseClassifier)
         and a decision tree classifier as final estimator.
     n_estimators : integer, optional (default=200)
         The number of trees in the forest.
-    criterion : string, optional (default="entropy")
-        The function to measure the quality of a split. Supported criteria are
-        "gini" for the Gini impurity and "entropy" for the information gain.
-        Note: this parameter is tree-specific. Default is "entropy"
     max_depth : integer or None, optional (default=None)
         The maximum depth of the tree. If None, then nodes are expanded until
         all leaves are pure or until all leaves contain less than
@@ -114,9 +108,6 @@ class ComposableTimeSeriesForestClassifier(BaseTimeSeriesForest, BaseClassifier)
         left child, and ``N_t_R`` is the number of samples in the right child.
         ``N``, ``N_t``, ``N_t_R`` and ``N_t_L`` all refer to the weighted sum,
         if ``sample_weight`` is passed.
-    min_impurity_split : float or None, (default=None)
-        Threshold for early stopping in tree growth. A node will split
-        if its impurity is above the threshold, otherwise it is a leaf.
     bootstrap : boolean, optional (default=False)
         Whether bootstrap samples are used when building trees.
     oob_score : bool (default=False)
@@ -195,7 +186,6 @@ def __init__(
         self,
         estimator=None,
         n_estimators=100,
-        criterion="entropy",
         max_depth=None,
         min_samples_split=2,
         min_samples_leaf=1,
@@ -203,7 +193,6 @@ def __init__(
         max_features=None,
         max_leaf_nodes=None,
         min_impurity_decrease=0.0,
-        min_impurity_split=None,
         bootstrap=False,
         oob_score=False,
         n_jobs=None,
@@ -218,7 +207,6 @@ def __init__(
 
         # Assign values, even though passed on to base estimator below,
         # necessary here for cloning
-        self.criterion = criterion
         self.max_depth = max_depth
         self.min_samples_split = min_samples_split
         self.min_samples_leaf = min_samples_leaf
@@ -226,7 +214,6 @@ def __init__(
         self.max_features = max_features
         self.max_leaf_nodes = max_leaf_nodes
         self.min_impurity_decrease = min_impurity_decrease
-        self.min_impurity_split = min_impurity_split
         self.max_samples = max_samples
 
         # Pass on params.
@@ -291,7 +278,6 @@ def _validate_estimator(self):
 
         # Set parameters according to naming in pipeline
         estimator_params = {
-            "criterion": self.criterion,
             "max_depth": self.max_depth,
             "min_samples_split": self.min_samples_split,
             "min_samples_leaf": self.min_samples_leaf,
@@ -299,7 +285,6 @@ def _validate_estimator(self):
             "max_features": self.max_features,
             "max_leaf_nodes": self.max_leaf_nodes,
             "min_impurity_decrease": self.min_impurity_decrease,
-            "min_impurity_split": self.min_impurity_split,
         }
         final_estimator = self.estimator_.steps[-1][0]
         self.estimator_params = {
@@ -312,18 +297,20 @@ def _validate_estimator(self):
             self.__setattr__(pname, pval)
 
     def predict(self, X):
-        """
-        Predict class for X.
+        """Predict class for X.
+
         The predicted class of an input sample is a vote by the trees in
         the forest, weighted by their probability estimates. That is,
         the predicted class is the one with highest mean probability
         estimate across the trees.
+
         Parameters
         ----------
         X : array-like or sparse matrix of shape (n_samples, n_features)
             The input samples. Internally, its dtype will be converted to
             ``dtype=np.float32``. If a sparse matrix is provided, it will be
             converted into a sparse ``csr_matrix``.
+
         Returns
         -------
         y : array-like of shape (n_samples,) or (n_samples, n_outputs)
@@ -348,17 +335,19 @@ def predict(self, X):
             return predictions
 
     def predict_log_proba(self, X):
-        """
-        Predict class log-probabilities for X.
+        """Predict class log-probabilities for X.
+
         The predicted class log-probabilities of an input sample is computed as
         the log of the mean predicted class probabilities of the trees in the
         forest.
+
         Parameters
         ----------
         X : array-like or sparse matrix of shape (n_samples, n_features)
             The input samples. Internally, its dtype will be converted to
             ``dtype=np.float32``. If a sparse matrix is provided, it will be
             converted into a sparse ``csr_matrix``.
+
         Returns
         -------
         p : array of shape (n_samples, n_classes), or a list of n_outputs
@@ -379,17 +368,19 @@ def predict_log_proba(self, X):
 
     def predict_proba(self, X):
         """Predict class probabilities for X.
+
         The predicted class probabilities of an input sample are computed as
         the mean predicted class probabilities of the trees in the forest. The
         class probability of a single tree is the fraction of samples of the
-        same
-        class in a leaf.
+        same class in a leaf.
+
         Parameters
         ----------
         X : array-like or sparse matrix of shape = [n_samples, n_features]
             The input samples. Internally, its dtype will be converted to
             ``dtype=np.float32``. If a sparse matrix is provided, it will be
             converted into a sparse ``csr_matrix``.
+
         Returns
         -------
         p : array of shape = [n_samples, n_classes], or a list of n_outputs
@@ -413,7 +404,7 @@ class in a leaf.
         return np.sum(all_proba, axis=0) / len(self.estimators_)
 
     def _set_oob_score(self, X, y):
-        """Compute out-of-bag score"""
+        """Compute out-of-bag score."""
         check_X_y(X, y)
         check_X(X, enforce_univariate=True)
 
@@ -460,6 +451,10 @@ def _set_oob_score(self, X, y):
 
         self.oob_score_ = oob_score / self.n_outputs_
 
+    # TODO - Implement this abstract method properly.
+    def _set_oob_score_and_attributes(self, X, y):
+        raise NotImplementedError("Not implemented.")
+
     def _validate_y_class_weight(self, y):
         check_classification_targets(y)
 
diff --git a/sktime/series_as_features/base/estimators/_ensemble.py b/sktime/series_as_features/base/estimators/_ensemble.py
--- a/sktime/series_as_features/base/estimators/_ensemble.py
+++ b/sktime/series_as_features/base/estimators/_ensemble.py
@@ -7,30 +7,25 @@
 __all__ = ["BaseTimeSeriesForest"]
 
 from abc import abstractmethod
-from warnings import catch_warnings
-from warnings import simplefilter
-from warnings import warn
+from warnings import catch_warnings, simplefilter, warn
 
 import numpy as np
 import pandas as pd
-from joblib import Parallel
-from joblib import delayed
+from joblib import Parallel, delayed
 from numpy import float64 as DOUBLE
 from scipy.sparse import issparse
 from sklearn.base import clone
 from sklearn.ensemble._base import _set_random_states
-from sklearn.ensemble._forest import BaseForest
-from sklearn.ensemble._forest import MAX_INT
-from sklearn.ensemble._forest import _generate_sample_indices
-from sklearn.ensemble._forest import _get_n_samples_bootstrap
+from sklearn.ensemble._forest import (
+    MAX_INT,
+    BaseForest,
+    _generate_sample_indices,
+    _get_n_samples_bootstrap,
+)
 from sklearn.exceptions import DataConversionWarning
-from sklearn.utils import check_array
-from sklearn.utils import check_random_state
-from sklearn.utils import compute_sample_weight
+from sklearn.utils import check_array, check_random_state, compute_sample_weight
 
-from sktime.transformations.panel.summarize import (
-    RandomIntervalFeatureExtractor,
-)
+from sktime.transformations.panel.summarize import RandomIntervalFeatureExtractor
 from sktime.utils.validation.panel import check_X_y
 
 
@@ -164,7 +159,7 @@ def fit(self, X, y, sample_weight=None):
 
         # Remap output
         self.n_columns = X.shape[1]
-        self.n_features_ = X.shape[1] if X.ndim == 2 else 1
+        self.n_features = X.shape[1] if X.ndim == 2 else 1
 
         y = np.atleast_1d(y)
         if y.ndim == 2 and y.shape[1] == 1:


Test patch: