Repository: sktime/sktime

Issue:
[BUG] 'kneighbors' method of KNeighborsTimeSeriesClassifier results in TypingError
**Describe the bug**
After a KNeighborsTimeSeriesClassifier has been successfully fit and successfully used to classify toy data, an attempt to use the 'kneighbors' method fails due to a TypingError. The method should return the indices of and distances to the neighbors of each point. 

**To Reproduce**
```python
from sktime.classification.distance_based import KNeighborsTimeSeriesClassifier
from sktime.datatypes import convert_to

Xtrain = convert_to(pd.DataFrame(data = {
    'sensor_a': [pd.Series([1,1,1]), pd.Series([2,2,2]), pd.Series([3,3,3])],
    'sensor_b': [pd.Series([4,4,4]), pd.Series([5,5,5]), pd.Series([6,6,6])],
    'sensor_c': [pd.Series([7,7,7]), pd.Series([8,8,8]), pd.Series([9,9,9])],
}), 'pd-multiindex')
ytrain = pd.Series(['label_1','label_2','label_3'])

kntsc = KNeighborsTimeSeriesClassifier(n_neighbors = 1)
kntsc.fit(Xtrain, ytrain)

Xtest = convert_to(pd.DataFrame({
    'sensor_a': [pd.Series([2.1,2,2]), pd.Series([1.1,1,1]), pd.Series([3.1,3,3])],
    'sensor_b': [pd.Series([5.1,5,5]), pd.Series([4.1,4,4]), pd.Series([6.1,6,6])],
    'sensor_c': [pd.Series([8.1,8,8]), pd.Series([7.1,7,7]), pd.Series([9.1,9,9])]
}), 'pd-multiindex')

display(kntsc.predict(Xtest))

print(kntsc.kneighbors(Xtest))
```

**Expected behavior**
The final line should print the indices of and distances to the neighbors of each point. 

**Additional context**
The 'fit' and 'predict' function correctly. The 'kneighbors' method returns the equivalent of an intermediate step of the 'predict' method.

The code provided has data in the 'pd-multiindex' mtype. The original code was written for data in 'nested_univ' mtype, and removing the 'convert_to' will demonstrate that the 'kneighbors' method doesn't work for 'nested_univ' mtype either.

I had used the 'kneighbors' method of sklearn.neighbors.KNeighborsClassifier in the code that I have been refactoring to use with sktime. The panel data format in my previous code was my own inefficient design and the code I wrote to handle it with sklearn pipelines is too lengthy and convoluted to provide here. However, sktime's KNeighborsTimeSeriesClassifier is described in the DocString as "An adapted version of the scikit-learn KNeighborsClassifier for time series data," so it should function the same way. 

This is the call stack trace: (also, each time the Jupyter notebook cell is executed, the last two lines are repeated an additional time)
```python
---------------------------------------------------------------------------
TypingError                               Traceback (most recent call last)
Cell In [43], line 31
     23 Xtest = convert_to(pd.DataFrame({
     24     'sensor_a': [pd.Series([2.1,2,2]), pd.Series([1.1,1,1]), pd.Series([3.1,3,3])],
     25     'sensor_b': [pd.Series([5.1,5,5]), pd.Series([4.1,4,4]), pd.Series([6.1,6,6])],
     26     'sensor_c': [pd.Series([8.1,8,8]), pd.Series([7.1,7,7]), pd.Series([9.1,9,9])]
     27 }), 'pd-multiindex')
     29 display(kntsc.predict(Xtest))
---> 31 kntsc.kneighbors(Xtest)

File ~\Anaconda3\envs\capstone_20221111\lib\site-packages\sktime\classification\distance_based\_time_series_neighbors.py:267, in KNeighborsTimeSeriesClassifier.kneighbors(self, X, n_neighbors, return_distance)
    264 self.check_is_fitted()
    266 # self._X should be the stored _X
--> 267 dist_mat = self._distance(X, self._X)
    269 result = self.knn_estimator_.kneighbors(
    270     dist_mat, n_neighbors=n_neighbors, return_distance=return_distance
    271 )
    273 # result is either dist, or (dist, ind) pair, depending on return_distance

File ~\Anaconda3\envs\capstone_20221111\lib\site-packages\sktime\classification\distance_based\_time_series_neighbors.py:199, in KNeighborsTimeSeriesClassifier._distance(self, X, X2)
    196     distance_params = {}
    198 if isinstance(distance, str):
--> 199     return pairwise_distance(X, X2, distance, **distance_params)
    200 else:
    201     if X2 is not None:

File ~\Anaconda3\envs\capstone_20221111\lib\site-packages\sktime\distances\_distance.py:2050, in pairwise_distance(x, y, metric, **kwargs)
   1960 def pairwise_distance(
   1961     x: np.ndarray,
   1962     y: np.ndarray = None,
   (...)
   1971     **kwargs: Any,
   1972 ) -> np.ndarray:
   1973     """Compute the pairwise distance matrix between two time series.
   1974 
   1975     First the distance metric is 'resolved'. This means the metric that is passed
   (...)
   2048            [ 58., 256.]])
   2049     """
-> 2050     _x = _make_3d_series(x)
   2051     if y is None:
   2052         y = x

File ~\Anaconda3\envs\capstone_20221111\lib\site-packages\numba\core\dispatcher.py:468, in _DispatcherBase._compile_for_args(self, *args, **kws)
    464         msg = (f"{str(e).rstrip()} \n\nThis error may have been caused "
    465                f"by the following argument(s):\n{args_str}\n")
    466         e.patch_message(msg)
--> 468     error_rewrite(e, 'typing')
    469 except errors.UnsupportedError as e:
    470     # Something unsupported is present in the user code, add help info
    471     error_rewrite(e, 'unsupported_error')

File ~\Anaconda3\envs\capstone_20221111\lib\site-packages\numba\core\dispatcher.py:409, in _DispatcherBase._compile_for_args.<locals>.error_rewrite(e, issue_type)
    407     raise e
    408 else:
--> 409     raise e.with_traceback(None)

TypingError: Failed in nopython mode pipeline (step: nopython frontend)
non-precise type pyobject
During: typing of argument at C:\Users\wechs\Anaconda3\envs\capstone_20221111\lib\site-packages\sktime\distances\_numba_utils.py (34)

File "..\..\..\..\Anaconda3\envs\capstone_20221111\lib\site-packages\sktime\distances\_numba_utils.py", line 34:
def _make_3d_series(x: np.ndarray) -> np.ndarray:
    <source elided>
    """
    num_dims = x.ndim
    ^ 

This error may have been caused by the following argument(s):
- argument 0: Cannot determine Numba type of <class 'pandas.core.frame.DataFrame'> 
```

**Versions**
System:
    python: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:51:29) [MSC v.1929 64 bit (AMD64)]
executable: C:\Users\wechs\Anaconda3\envs\capstone_20221111\python.exe
   machine: Windows-10-10.0.19044-SP0

Python dependencies:
          pip: 22.3.1
   setuptools: 65.5.1
      sklearn: 1.1.3
       sktime: 0.14.0
  statsmodels: 0.13.5
        numpy: 1.23.3
        scipy: 1.9.3
       pandas: 1.4.4
   matplotlib: 3.5.3
       joblib: 1.2.0
        numba: 0.56.3
     pmdarima: 2.0.1
      tsfresh: 0.19.0

<!-- Thanks for contributing! -->


Comments:

Patch:
diff --git a/docs/source/get_started.rst b/docs/source/get_started.rst
index ce8ab2dd6..0e6ffc82d 100644
--- a/docs/source/get_started.rst
+++ b/docs/source/get_started.rst
@@ -124,7 +124,7 @@ Time Series Clustering
 
     >>> from sklearn.model_selection import train_test_split
     >>> from sktime.clustering.k_means import TimeSeriesKMeans
-    >>> from sktime.clustering.evaluation._plot_clustering import plot_cluster_algorithm
+    >>> from sktime.clustering.utils.plotting._plot_partitions import plot_cluster_algorithm
     >>> from sktime.datasets import load_arrow_head
 
     >>> X, y = load_arrow_head()
diff --git a/sktime/classification/distance_based/_time_series_neighbors.py b/sktime/classification/distance_based/_time_series_neighbors.py
index 04eabd93f..cf5d91cd7 100644
--- a/sktime/classification/distance_based/_time_series_neighbors.py
+++ b/sktime/classification/distance_based/_time_series_neighbors.py
@@ -263,6 +263,9 @@ class KNeighborsTimeSeriesClassifier(BaseClassifier):
         """
         self.check_is_fitted()
 
+        # boilerplate input checks for predict-like methods
+        X = self._check_convert_X_for_predict(X)
+
         # self._X should be the stored _X
         dist_mat = self._distance(X, self._X)
 
diff --git a/sktime/regression/distance_based/_time_series_neighbors.py b/sktime/regression/distance_based/_time_series_neighbors.py
index 9e286cb2d..7d8e41290 100644
--- a/sktime/regression/distance_based/_time_series_neighbors.py
+++ b/sktime/regression/distance_based/_time_series_neighbors.py
@@ -210,6 +210,11 @@ class KNeighborsTimeSeriesRegressor(BaseRegressor):
         ind : array
             Indices of the nearest points in the population matrix.
         """
+        self.check_is_fitted()
+
+        # boilerplate input checks for predict-like methods
+        X = self._check_convert_X_for_predict(X)
+
         # self._X should be the stored _X
         dist_mat = self._distance(X, self._X)
 
diff --git a/sktime/transformations/series/detrend/_detrend.py b/sktime/transformations/series/detrend/_detrend.py
index b1a32cae9..4a5e84eca 100644
--- a/sktime/transformations/series/detrend/_detrend.py
+++ b/sktime/transformations/series/detrend/_detrend.py
@@ -4,7 +4,7 @@
 """Implements transformations to detrend a time series."""
 
 __all__ = ["Detrender"]
-__author__ = ["mloning", "SveaMeyer13"]
+__author__ = ["mloning", "SveaMeyer13", "KishManani"]
 
 import pandas as pd
 
@@ -38,6 +38,11 @@ class Detrender(BaseTransformer):
         The forecasting model to remove the trend with
             (e.g. PolynomialTrendForecaster).
         If forecaster is None, PolynomialTrendForecaster(degree=1) is used.
+    model : {"additive", "multiplicative"}, default="additive"
+        If `model="additive"` the `forecaster` is fit to the original time
+        series and the `transform` method subtracts the trend from the time series.
+        If `model="multiplicative"` the `forecaster` is fit to the original time
+        series and the `transform` method divides the trend from the time series.
 
     Attributes
     ----------
@@ -74,8 +79,9 @@ class Detrender(BaseTransformer):
         "transform-returns-same-time-index": True,
     }
 
-    def __init__(self, forecaster=None):
+    def __init__(self, forecaster=None, model="additive"):
         self.forecaster = forecaster
+        self.model = model
         self.forecaster_ = None
         super(Detrender, self).__init__()
 
@@ -118,6 +124,10 @@ class Detrender(BaseTransformer):
         else:
             raise TypeError("X must be pd.Series or pd.DataFrame")
 
+        allowed_models = ("additive", "multiplicative")
+        if self.model not in allowed_models:
+            raise ValueError("`model` must be 'additive' or 'multiplicative'")
+
         return self
 
     def _transform(self, X, y=None):
@@ -143,8 +153,10 @@ class Detrender(BaseTransformer):
         if isinstance(X, pd.Series):
             # note: the y in the transformer is exogeneous in the forecaster, i.e., X
             X_pred = self.forecaster_.predict(fh=fh, X=y)
-            Xt = X - X_pred
-            return Xt
+            if self.model == "additive":
+                return X - X_pred
+            elif self.model == "multiplicative":
+                return X / X_pred
         # multivariate: X is pd.DataFrame
         elif isinstance(X, pd.DataFrame):
             Xt = X.copy()
@@ -159,7 +171,10 @@ class Detrender(BaseTransformer):
                 )
             for colname in Xt.columns:
                 X_pred = self.forecaster_[colname].predict(fh=fh, X=y)
-                Xt[colname] = Xt[colname] - X_pred
+                if self.model == "additive":
+                    Xt[colname] = Xt[colname] - X_pred
+                elif self.model == "multiplicative":
+                    Xt[colname] = Xt[colname] / X_pred
             return Xt
         else:
             raise TypeError("X must be pd.Series or pd.DataFrame")
@@ -185,7 +200,10 @@ class Detrender(BaseTransformer):
         if isinstance(X, pd.Series):
             # note: the y in the transformer is exogeneous in the forecaster, i.e., X
             X_pred = self.forecaster_.predict(fh=fh, X=y)
-            return X + X_pred
+            if self.model == "additive":
+                return X + X_pred
+            elif self.model == "multiplicative":
+                return X * X_pred
         # multivariate: X is pd.DataFrame
         if isinstance(X, pd.DataFrame):
             X = X.copy()
@@ -200,7 +218,11 @@ class Detrender(BaseTransformer):
                 )
             for colname in X.columns:
                 X_pred = self.forecaster_[colname].predict(fh=fh, X=y)
-                X[colname] = X[colname] + X_pred
+                if self.model == "additive":
+                    X[colname] = X[colname] + X_pred
+                elif self.model == "multiplicative":
+                    X[colname] = X[colname] * X_pred
+
             return X
 
     def _update(self, X, y=None, update_params=True):


Test patch:
diff --git a/sktime/classification/distance_based/tests/test_time_series_neighbors.py b/sktime/classification/distance_based/tests/test_time_series_neighbors.py
index 60bf880ea..de7afd851 100644
--- a/sktime/classification/distance_based/tests/test_time_series_neighbors.py
+++ b/sktime/classification/distance_based/tests/test_time_series_neighbors.py
@@ -1,6 +1,7 @@
 # -*- coding: utf-8 -*-
-"""Test function of elastic distance nearest neighbour classifiers."""
+"""Tests for KNeighborsTimeSeriesClassifier."""
 import numpy as np
+import pandas as pd
 import pytest
 
 from sktime.alignment.dtw_python import AlignerDTW
@@ -109,3 +110,26 @@ def test_knn_with_aggrdistance():
     clf = KNeighborsTimeSeriesClassifier(distance=aggr_dist)
 
     clf.fit(X, y)
+
+
+def test_knn_kneighbors():
+    """Tests kneighbors method and absence of bug #3798."""
+    from sktime.utils._testing.hierarchical import _make_hierarchical
+
+    Xtrain = _make_hierarchical(hierarchy_levels=(3,), n_columns=3)
+    Xtest = _make_hierarchical(hierarchy_levels=(5,), n_columns=3)
+
+    ytrain = pd.Series(["label_1", "label_2", "label_3"])
+
+    kntsc = KNeighborsTimeSeriesClassifier(n_neighbors=1)
+    kntsc.fit(Xtrain, ytrain)
+
+    ret = kntsc.kneighbors(Xtest)
+    assert isinstance(ret, tuple)
+    assert len(ret) == 2
+
+    dist, ind = ret
+    assert isinstance(dist, np.ndarray)
+    assert dist.shape == (5, 1)
+    assert isinstance(ind, np.ndarray)
+    assert ind.shape == (5, 1)
diff --git a/sktime/regression/distance_based/tests/__init__.py b/sktime/regression/distance_based/tests/__init__.py
new file mode 100644
index 000000000..18fca48ad
--- /dev/null
+++ b/sktime/regression/distance_based/tests/__init__.py
@@ -0,0 +1,2 @@
+# -*- coding: utf-8 -*-
+"""Distance based test code."""
diff --git a/sktime/regression/distance_based/tests/test_time_series_neighbors.py b/sktime/regression/distance_based/tests/test_time_series_neighbors.py
new file mode 100644
index 000000000..6be87cc9c
--- /dev/null
+++ b/sktime/regression/distance_based/tests/test_time_series_neighbors.py
@@ -0,0 +1,31 @@
+# -*- coding: utf-8 -*-
+"""Tests for KNeighborsTimeSeriesRegressor."""
+import numpy as np
+import pandas as pd
+
+from sktime.regression.distance_based._time_series_neighbors import (
+    KNeighborsTimeSeriesRegressor,
+)
+
+
+def test_knn_kneighbors():
+    """Tests kneighbors method and absence of bug #3798."""
+    from sktime.utils._testing.hierarchical import _make_hierarchical
+
+    Xtrain = _make_hierarchical(hierarchy_levels=(3,), n_columns=3)
+    Xtest = _make_hierarchical(hierarchy_levels=(5,), n_columns=3)
+
+    ytrain = pd.Series([1, 1.5, 2])
+
+    kntsc = KNeighborsTimeSeriesRegressor(n_neighbors=1)
+    kntsc.fit(Xtrain, ytrain)
+
+    ret = kntsc.kneighbors(Xtest)
+    assert isinstance(ret, tuple)
+    assert len(ret) == 2
+
+    dist, ind = ret
+    assert isinstance(dist, np.ndarray)
+    assert dist.shape == (5, 1)
+    assert isinstance(ind, np.ndarray)
+    assert ind.shape == (5, 1)
diff --git a/sktime/transformations/series/detrend/tests/test_detrend.py b/sktime/transformations/series/detrend/tests/test_detrend.py
index 4641a3427..50ff9e676 100644
--- a/sktime/transformations/series/detrend/tests/test_detrend.py
+++ b/sktime/transformations/series/detrend/tests/test_detrend.py
@@ -1,17 +1,29 @@
 # -*- coding: utf-8 -*-
 """Test detrenders."""
+import numpy as np
+import pandas as pd
+import pytest
 
-__author__ = ["mloning"]
+from sktime.datasets import load_airline
+from sktime.forecasting.tests.test_trend import get_expected_polynomial_coefs
+from sktime.forecasting.trend import PolynomialTrendForecaster
+from sktime.transformations.series.detrend import Detrender
+
+__author__ = ["mloning", "KishManani"]
 __all__ = []
 
 
-def test_polynomial_detrending():
-    import numpy as np
-    import pandas as pd
+@pytest.fixture()
+def y_series():
+    return load_airline()
+
 
-    from sktime.forecasting.tests.test_trend import get_expected_polynomial_coefs
-    from sktime.forecasting.trend import PolynomialTrendForecaster
-    from sktime.transformations.series.detrend import Detrender
+@pytest.fixture()
+def y_dataframe():
+    return load_airline().to_frame()
+
+
+def test_polynomial_detrending():
 
     y = pd.Series(np.arange(20) * 0.5) + np.random.normal(0, 1, size=20)
     forecaster = PolynomialTrendForecaster(degree=1, with_intercept=True)
@@ -34,3 +46,79 @@ def test_polynomial_detrending():
     actual = transformer.transform(y)
     expected = y - expected_trend
     np.testing.assert_array_almost_equal(actual, expected)
+
+
+def test_multiplicative_detrending_series(y_series):
+    """Tests we get the expected result when setting `model=multiplicative`."""
+    # Load test dataset
+    y = y_series
+
+    # Get the trend
+    forecaster = PolynomialTrendForecaster(degree=1, with_intercept=True)
+    trend = forecaster.fit_predict(y, fh=y.index)
+
+    # De-trend the time series
+    detrender = Detrender(forecaster, model="multiplicative")
+    y_transformed = detrender.fit_transform(y)
+
+    # Compute the expected de-trended time series
+    expected = y / trend
+
+    pd.testing.assert_series_equal(y_transformed, expected)
+
+
+def test_multiplicative_detrending_dataframe(y_dataframe):
+    """Tests we get the expected result when setting `model=multiplicative`."""
+    # Load test dataset
+    y = y_dataframe
+
+    # Get the trend
+    forecaster = PolynomialTrendForecaster(degree=1, with_intercept=True)
+    trend = forecaster.fit_predict(y, fh=y.index)
+
+    # De-trend the time series
+    detrender = Detrender(forecaster, model="multiplicative")
+    y_transformed = detrender.fit_transform(y)
+
+    # Compute the expected de-trended time series
+    expected = y / trend
+
+    pd.testing.assert_frame_equal(y_transformed, expected)
+
+
+def test_additive_detrending_series(y_series):
+    """Tests we get the expected result when setting `model=additive`."""
+    # Load test dataset
+    y = y_series
+
+    # Get the trend
+    forecaster = PolynomialTrendForecaster(degree=1, with_intercept=True)
+    trend = forecaster.fit_predict(y, fh=y.index)
+
+    # De-trend the time series
+    detrender = Detrender(forecaster, model="additive")
+    y_transformed = detrender.fit_transform(y)
+
+    # Compute the expected de-trended time series
+    expected = y - trend
+
+    pd.testing.assert_series_equal(y_transformed, expected)
+
+
+def test_additive_detrending_dataframe(y_dataframe):
+    """Tests we get the expected result when setting `model=additive`."""
+    # Load test dataset
+    y = y_dataframe
+
+    # Get the trend
+    forecaster = PolynomialTrendForecaster(degree=1, with_intercept=True)
+    trend = forecaster.fit_predict(y, fh=y.index)
+
+    # De-trend the time series
+    detrender = Detrender(forecaster, model="additive")
+    y_transformed = detrender.fit_transform(y)
+
+    # Compute the expected de-trended time series
+    expected = y - trend
+
+    pd.testing.assert_frame_equal(y_transformed, expected)