Repository: sktime/sktime

Issue:
Cannot access feature_importances_ attribute for RISE estimator 
**Describe the bug**
<!--
-->
Cannot access feature_importances_ attribute for RISE estimator without error.


**To Reproduce**
<!--
Add a Minimal, Complete, and Verifiable example (for more details, see e.g. https://stackoverflow.com/help/mcve

If the code is too long, feel free to put it in a public gist and link it in the issue: https://gist.github.com
-->

```python
from sklearn.model_selection import train_test_split
from sktime.datasets import load_arrow_head
from sktime.classification.frequency_based import RandomIntervalSpectralForest

X, y = load_arrow_head(return_X_y=True)
X_train, X_test, y_train, y_test = train_test_split(X, y)

rise = RandomIntervalSpectralForest(n_estimators=10)
rise.fit(X_train, y_train)
rise.score(X_test, y_test)
rise.feature_importances_

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
TypeError: only size-1 arrays can be converted to Python scalars

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
<ipython-input-26-4a0a11fa4729> in <module>
      5 rise.score(X_test, y_test)
      6 
----> 7 rise.feature_importances_

~/.local/lib/python3.6/site-packages/sklearn/ensemble/_forest.py in feature_importances_(self)
    453 
    454         all_importances = np.mean(all_importances,
--> 455                                   axis=0, dtype=np.float64)
    456         return all_importances / np.sum(all_importances)
    457 

<__array_function__ internals> in mean(*args, **kwargs)

~/.local/lib/python3.6/site-packages/numpy/core/fromnumeric.py in mean(a, axis, dtype, out, keepdims)
   3371 
   3372     return _methods._mean(a, axis=axis, dtype=dtype,
-> 3373                           out=out, **kwargs)
   3374 
   3375 

~/.local/lib/python3.6/site-packages/numpy/core/_methods.py in _mean(a, axis, dtype, out, keepdims)
    158             is_float16_result = True
    159 
--> 160     ret = umr_sum(arr, axis, dtype, out, keepdims)
    161     if isinstance(ret, mu.ndarray):
    162         ret = um.true_divide(

ValueError: setting an array element with a sequence.
```

**Expected behavior**
<!--
-->
rise.feature_importances_ should be a pandas dataframe. 


**Additional context**
<!--
"Add any other context about the problem here."
-->

**Versions**
<details>

<!--
Please run the following code snippet and paste the output here:
 
from sktime import show_versions; show_versions()
-->

</details>

<!-- Thanks for contributing! -->



Discussion:
Hi @KoolSkotcheCoding, thanks for raising the issue! 

We haven't implemented feature importances for RISE yet, there is an experimental feature importance graph method for the `TimeSeriesForestClassifier` in `sktime.classification.compose`. 

We need to override the method we inherit from sklearn and raise a NotImplementedError - would appreciate a PR! 
Would it be better if we could compute the `feature_importances_`, which enables one to find the most discriminative timestamps (if I understand correctly)

A little thought on implementation:
Since RISE uses the decision tree in scikit-learn as the base estimator, which is already equipped with the property `feature_importances_`, the implementation of `feature_importances_` for RISE simply needs to add another function decorated by `@property`.

Any discussion is welcomed : )
Hi @AaronX121, yes you're right about the property decorator! We have an implementation for time series forest [here](https://github.com/alan-turing-institute/sktime/blob/cc733340e697bd20b7612783e0c03c016da9e18c/sktime/series_as_features/base/estimators/_ensemble.py#L292). 

Compared to scikit-learn, the main complication is that we need to keep track of the intervals too when aggregating feature importances. More details in section 4.4 in the [original paper](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.743.4832&rep=rep1&type=pdf).

Resolving patch:
diff --git a/sktime/classification/frequency_based/_rise.py b/sktime/classification/frequency_based/_rise.py
--- a/sktime/classification/frequency_based/_rise.py
+++ b/sktime/classification/frequency_based/_rise.py
@@ -161,6 +161,13 @@ def __init__(
         # We need to add is-fitted state when inheriting from scikit-learn
         self._is_fitted = False
 
+    @property
+    def feature_importances_(self):
+        raise NotImplementedError(
+            "The impurity-based feature importances of "
+            "RandomIntervalSpectralForest is currently not supported."
+        )
+
     def fit(self, X, y):
         """
         Build a forest of trees from the training set (X, y) using random
@@ -340,10 +347,10 @@ def acf(x, max_lag):
         y[lag - 1] = y[lag - 1] / (length - lag)
         v1 = ss1 / (length - lag) - s1 * s1
         v2 = ss2 / (length - lag) - s2 * s2
-        if v1 <= 0.000000001 and v2 <= 0.000000001:  # Both zero variance,
+        if v1 <= 1e-9 and v2 <= 1e-9:  # Both zero variance,
             # so must be 100% correlated
             y[lag - 1] = 1
-        elif v1 <= 0.000000001 or v2 <= 0.000000001:  # One zero variance
+        elif v1 <= 1e-9 or v2 <= 1e-9:  # One zero variance
             # the other not
             y[lag - 1] = 0
         else:


Test patch: