Repository: sktime/sktime

Issue:
[BUG] 'kneighbors' method of KNeighborsTimeSeriesClassifier results in TypingError
**Describe the bug**
After a KNeighborsTimeSeriesClassifier has been successfully fit and successfully used to classify toy data, an attempt to use the 'kneighbors' method fails due to a TypingError. The method should return the indices of and distances to the neighbors of each point. 

**To Reproduce**
```python
from sktime.classification.distance_based import KNeighborsTimeSeriesClassifier
from sktime.datatypes import convert_to

Xtrain = convert_to(pd.DataFrame(data = {
    'sensor_a': [pd.Series([1,1,1]), pd.Series([2,2,2]), pd.Series([3,3,3])],
    'sensor_b': [pd.Series([4,4,4]), pd.Series([5,5,5]), pd.Series([6,6,6])],
    'sensor_c': [pd.Series([7,7,7]), pd.Series([8,8,8]), pd.Series([9,9,9])],
}), 'pd-multiindex')
ytrain = pd.Series(['label_1','label_2','label_3'])

kntsc = KNeighborsTimeSeriesClassifier(n_neighbors = 1)
kntsc.fit(Xtrain, ytrain)

Xtest = convert_to(pd.DataFrame({
    'sensor_a': [pd.Series([2.1,2,2]), pd.Series([1.1,1,1]), pd.Series([3.1,3,3])],
    'sensor_b': [pd.Series([5.1,5,5]), pd.Series([4.1,4,4]), pd.Series([6.1,6,6])],
    'sensor_c': [pd.Series([8.1,8,8]), pd.Series([7.1,7,7]), pd.Series([9.1,9,9])]
}), 'pd-multiindex')

display(kntsc.predict(Xtest))

print(kntsc.kneighbors(Xtest))
```

**Expected behavior**
The final line should print the indices of and distances to the neighbors of each point. 

**Additional context**
The 'fit' and 'predict' function correctly. The 'kneighbors' method returns the equivalent of an intermediate step of the 'predict' method.

The code provided has data in the 'pd-multiindex' mtype. The original code was written for data in 'nested_univ' mtype, and removing the 'convert_to' will demonstrate that the 'kneighbors' method doesn't work for 'nested_univ' mtype either.

I had used the 'kneighbors' method of sklearn.neighbors.KNeighborsClassifier in the code that I have been refactoring to use with sktime. The panel data format in my previous code was my own inefficient design and the code I wrote to handle it with sklearn pipelines is too lengthy and convoluted to provide here. However, sktime's KNeighborsTimeSeriesClassifier is described in the DocString as "An adapted version of the scikit-learn KNeighborsClassifier for time series data," so it should function the same way. 

This is the call stack trace: (also, each time the Jupyter notebook cell is executed, the last two lines are repeated an additional time)
```python
---------------------------------------------------------------------------
TypingError                               Traceback (most recent call last)
Cell In [43], line 31
     23 Xtest = convert_to(pd.DataFrame({
     24     'sensor_a': [pd.Series([2.1,2,2]), pd.Series([1.1,1,1]), pd.Series([3.1,3,3])],
     25     'sensor_b': [pd.Series([5.1,5,5]), pd.Series([4.1,4,4]), pd.Series([6.1,6,6])],
     26     'sensor_c': [pd.Series([8.1,8,8]), pd.Series([7.1,7,7]), pd.Series([9.1,9,9])]
     27 }), 'pd-multiindex')
     29 display(kntsc.predict(Xtest))
---> 31 kntsc.kneighbors(Xtest)

File ~\Anaconda3\envs\capstone_20221111\lib\site-packages\sktime\classification\distance_based\_time_series_neighbors.py:267, in KNeighborsTimeSeriesClassifier.kneighbors(self, X, n_neighbors, return_distance)
    264 self.check_is_fitted()
    266 # self._X should be the stored _X
--> 267 dist_mat = self._distance(X, self._X)
    269 result = self.knn_estimator_.kneighbors(
    270     dist_mat, n_neighbors=n_neighbors, return_distance=return_distance
    271 )
    273 # result is either dist, or (dist, ind) pair, depending on return_distance

File ~\Anaconda3\envs\capstone_20221111\lib\site-packages\sktime\classification\distance_based\_time_series_neighbors.py:199, in KNeighborsTimeSeriesClassifier._distance(self, X, X2)
    196     distance_params = {}
    198 if isinstance(distance, str):
--> 199     return pairwise_distance(X, X2, distance, **distance_params)
    200 else:
    201     if X2 is not None:

File ~\Anaconda3\envs\capstone_20221111\lib\site-packages\sktime\distances\_distance.py:2050, in pairwise_distance(x, y, metric, **kwargs)
   1960 def pairwise_distance(
   1961     x: np.ndarray,
   1962     y: np.ndarray = None,
   (...)
   1971     **kwargs: Any,
   1972 ) -> np.ndarray:
   1973     """Compute the pairwise distance matrix between two time series.
   1974 
   1975     First the distance metric is 'resolved'. This means the metric that is passed
   (...)
   2048            [ 58., 256.]])
   2049     """
-> 2050     _x = _make_3d_series(x)
   2051     if y is None:
   2052         y = x

File ~\Anaconda3\envs\capstone_20221111\lib\site-packages\numba\core\dispatcher.py:468, in _DispatcherBase._compile_for_args(self, *args, **kws)
    464         msg = (f"{str(e).rstrip()} \n\nThis error may have been caused "
    465                f"by the following argument(s):\n{args_str}\n")
    466         e.patch_message(msg)
--> 468     error_rewrite(e, 'typing')
    469 except errors.UnsupportedError as e:
    470     # Something unsupported is present in the user code, add help info
    471     error_rewrite(e, 'unsupported_error')

File ~\Anaconda3\envs\capstone_20221111\lib\site-packages\numba\core\dispatcher.py:409, in _DispatcherBase._compile_for_args.<locals>.error_rewrite(e, issue_type)
    407     raise e
    408 else:
--> 409     raise e.with_traceback(None)

TypingError: Failed in nopython mode pipeline (step: nopython frontend)
non-precise type pyobject
During: typing of argument at C:\Users\wechs\Anaconda3\envs\capstone_20221111\lib\site-packages\sktime\distances\_numba_utils.py (34)

File "..\..\..\..\Anaconda3\envs\capstone_20221111\lib\site-packages\sktime\distances\_numba_utils.py", line 34:
def _make_3d_series(x: np.ndarray) -> np.ndarray:
    <source elided>
    """
    num_dims = x.ndim
    ^ 

This error may have been caused by the following argument(s):
- argument 0: Cannot determine Numba type of <class 'pandas.core.frame.DataFrame'> 
```

**Versions**
System:
    python: 3.9.13 | packaged by conda-forge | (main, May 27 2022, 16:51:29) [MSC v.1929 64 bit (AMD64)]
executable: C:\Users\wechs\Anaconda3\envs\capstone_20221111\python.exe
   machine: Windows-10-10.0.19044-SP0

Python dependencies:
          pip: 22.3.1
   setuptools: 65.5.1
      sklearn: 1.1.3
       sktime: 0.14.0
  statsmodels: 0.13.5
        numpy: 1.23.3
        scipy: 1.9.3
       pandas: 1.4.4
   matplotlib: 3.5.3
       joblib: 1.2.0
        numba: 0.56.3
     pmdarima: 2.0.1
      tsfresh: 0.19.0

<!-- Thanks for contributing! -->



Discussion:

Resolving patch:
diff --git a/sktime/classification/distance_based/_time_series_neighbors.py b/sktime/classification/distance_based/_time_series_neighbors.py
--- a/sktime/classification/distance_based/_time_series_neighbors.py
+++ b/sktime/classification/distance_based/_time_series_neighbors.py
@@ -263,6 +263,9 @@ def kneighbors(self, X, n_neighbors=None, return_distance=True):
         """
         self.check_is_fitted()
 
+        # boilerplate input checks for predict-like methods
+        X = self._check_convert_X_for_predict(X)
+
         # self._X should be the stored _X
         dist_mat = self._distance(X, self._X)
 
diff --git a/sktime/regression/distance_based/_time_series_neighbors.py b/sktime/regression/distance_based/_time_series_neighbors.py
--- a/sktime/regression/distance_based/_time_series_neighbors.py
+++ b/sktime/regression/distance_based/_time_series_neighbors.py
@@ -210,6 +210,11 @@ def kneighbors(self, X, n_neighbors=None, return_distance=True):
         ind : array
             Indices of the nearest points in the population matrix.
         """
+        self.check_is_fitted()
+
+        # boilerplate input checks for predict-like methods
+        X = self._check_convert_X_for_predict(X)
+
         # self._X should be the stored _X
         dist_mat = self._distance(X, self._X)
 


Test patch:
diff --git a/sktime/classification/distance_based/tests/test_time_series_neighbors.py b/sktime/classification/distance_based/tests/test_time_series_neighbors.py
--- a/sktime/classification/distance_based/tests/test_time_series_neighbors.py
+++ b/sktime/classification/distance_based/tests/test_time_series_neighbors.py
@@ -1,6 +1,7 @@
 # -*- coding: utf-8 -*-
-"""Test function of elastic distance nearest neighbour classifiers."""
+"""Tests for KNeighborsTimeSeriesClassifier."""
 import numpy as np
+import pandas as pd
 import pytest
 
 from sktime.alignment.dtw_python import AlignerDTW
@@ -109,3 +110,26 @@ def test_knn_with_aggrdistance():
     clf = KNeighborsTimeSeriesClassifier(distance=aggr_dist)
 
     clf.fit(X, y)
+
+
+def test_knn_kneighbors():
+    """Tests kneighbors method and absence of bug #3798."""
+    from sktime.utils._testing.hierarchical import _make_hierarchical
+
+    Xtrain = _make_hierarchical(hierarchy_levels=(3,), n_columns=3)
+    Xtest = _make_hierarchical(hierarchy_levels=(5,), n_columns=3)
+
+    ytrain = pd.Series(["label_1", "label_2", "label_3"])
+
+    kntsc = KNeighborsTimeSeriesClassifier(n_neighbors=1)
+    kntsc.fit(Xtrain, ytrain)
+
+    ret = kntsc.kneighbors(Xtest)
+    assert isinstance(ret, tuple)
+    assert len(ret) == 2
+
+    dist, ind = ret
+    assert isinstance(dist, np.ndarray)
+    assert dist.shape == (5, 1)
+    assert isinstance(ind, np.ndarray)
+    assert ind.shape == (5, 1)
diff --git a/sktime/regression/distance_based/tests/__init__.py b/sktime/regression/distance_based/tests/__init__.py
new file mode 100644
--- /dev/null
+++ b/sktime/regression/distance_based/tests/__init__.py
@@ -0,0 +1,2 @@
+# -*- coding: utf-8 -*-
+"""Distance based test code."""
diff --git a/sktime/regression/distance_based/tests/test_time_series_neighbors.py b/sktime/regression/distance_based/tests/test_time_series_neighbors.py
new file mode 100644
--- /dev/null
+++ b/sktime/regression/distance_based/tests/test_time_series_neighbors.py
@@ -0,0 +1,31 @@
+# -*- coding: utf-8 -*-
+"""Tests for KNeighborsTimeSeriesRegressor."""
+import numpy as np
+import pandas as pd
+
+from sktime.regression.distance_based._time_series_neighbors import (
+    KNeighborsTimeSeriesRegressor,
+)
+
+
+def test_knn_kneighbors():
+    """Tests kneighbors method and absence of bug #3798."""
+    from sktime.utils._testing.hierarchical import _make_hierarchical
+
+    Xtrain = _make_hierarchical(hierarchy_levels=(3,), n_columns=3)
+    Xtest = _make_hierarchical(hierarchy_levels=(5,), n_columns=3)
+
+    ytrain = pd.Series([1, 1.5, 2])
+
+    kntsc = KNeighborsTimeSeriesRegressor(n_neighbors=1)
+    kntsc.fit(Xtrain, ytrain)
+
+    ret = kntsc.kneighbors(Xtest)
+    assert isinstance(ret, tuple)
+    assert len(ret) == 2
+
+    dist, ind = ret
+    assert isinstance(dist, np.ndarray)
+    assert dist.shape == (5, 1)
+    assert isinstance(ind, np.ndarray)
+    assert ind.shape == (5, 1)