Repository: scikit-learn/scikit-learn
Pull request: FIX Fixes KNeighborsRegressor.predict with array-likes
#### Reference Issues/PRs
Fixes #22683

#### What does this implement/fix? Explain your changes.
Previously `KenighborsRegressor.predict` was documented to accept "array-likes" however when `weights` is given as a callable, `predict(X)` calls `X.shape[0]`. To make `predict` compatible with array-likes this PR changes `X.shape[0]-->len(X)` and adds a non-regression test that fails on the main branch. 

#### Any other comments?

Problem statement:
KNeighborsRegressor with a callable weights stopped working with numpy 1.22.2
### Describe the bug

When you use a callable for the weights param you get:

AttributeError: 'list' object has no attribute 'shape'

`neigh = KNeighborsRegressor(n_neighbors=5, algorithm='brute', metric=euclidean_distance, weights=weights_p2)`

### Steps/Code to Reproduce

```
import numpy as np
from sklearn import neighbors
from sklearn.neighbors import KNeighborsRegressor


x = [5, 6, 4, 3, 4, 3, 1]
y = [2, 5, 2, 3, 7, 6, 8]
g = [4, 3, 3, 3, 2, 2, 1]

D = []
for i in range(len(x)):
    D.append([x[i],y[i]])

def _weights(dist):
    print(dist)
    
    n = len(dist[0])
    w = list()
    d = dist[0]
    
    for i in range(n):
        if d[i] <= 0.0000:
            print("DIV by 0")
            w = [0]*n
            w[i] = 1.0
            print("Weights: ", end='')
            print(w)
            return [w]
        else:
            w.append((1/d[i])**2)
            
    print("   Weights: ", end='')
    print(w)
        
    return [w]



neigh = KNeighborsRegressor(n_neighbors=5, algorithm='brute', metric=euclidean_distance, weights=_weights)
neigh.fit(D, g)
print(neigh.predict([[2, 4]]))
```

Change the weights to `weights='uniform'` (no callable) it will work with numpy 1.22 but with the callable weights I need to downgrade to 1.19.5

### Expected Results

[[1.41421356 2.23606798 2.82842712 3.60555128 3.60555128]]
   Weights: [0.4999999999999999, 0.19999999999999998, 0.12499999999999997, 0.07692307692307693, 0.07692307692307693]
[2.7956778]

### Actual Results

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Input In [54], in <cell line: 8>()
      6 neigh.fit(D, g)
      7 print("Expected GPA: ", end='')
----> 8 print(round(neigh.predict([[2, 4]])[0], 4))
      9 print()

File ~/opt/anaconda3/envs/tensorflow-3/lib/python3.10/site-packages/sklearn/neighbors/_regression.py:240, in KNeighborsRegressor.predict(self, X)
    238     y_pred = np.mean(_y[neigh_ind], axis=1)
    239 else:
--> 240     y_pred = np.empty((X.shape[0], _y.shape[1]), dtype=np.float64)
    241     denom = np.sum(weights, axis=1)
    243     for j in range(_y.shape[1]):

AttributeError: 'list' object has no attribute 'shape'


### Versions

```shell
/Users/ray/opt/anaconda3/envs/tensorflow-3/lib/python3.10/site-packages/_distutils_hack/__init__.py:30: UserWarning: Setuptools is replacing distutils.
  warnings.warn("Setuptools is replacing distutils.")

System:
    python: 3.10.2 | packaged by conda-forge | (main, Feb  1 2022, 19:30:18) [Clang 11.1.0 ]
executable: /Users/ray/opt/anaconda3/envs/tensorflow-3/bin/python3.10
   machine: macOS-12.2.1-x86_64-i386-64bit

Python dependencies:
          pip: 22.0.3
   setuptools: 60.9.3
      sklearn: 1.0.2
        numpy: 1.22.2
        scipy: 1.8.0
       Cython: None
       pandas: 1.4.1
   matplotlib: None
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
```

Hints from issue discussion:

Commit message: Added non-regression test and changelog entry

Patch to review:
diff --git a/doc/whats_new/v1.1.rst b/doc/whats_new/v1.1.rst
index ee0f75b92c789..3b8ec6ff4a6c8 100644
--- a/doc/whats_new/v1.1.rst
+++ b/doc/whats_new/v1.1.rst
@@ -656,6 +656,11 @@ Changelog
   instead of `__init__`. :pr:`21430` by :user:`Desislava Vasileva <DessyVV>` and
   :user:`Lucy Jimenez <LucyJimenez>`.
 
+- |Fix| :func:`neighbors.KNeighborsRegressor.predict` now works properly when
+  given an array-like input if `KNeighborsRegressor` is first constructed with a
+  callable passed to the `weights` parameter. :pr:`22687` by
+  :user:`Meekail Zain <micky774>`
+
 :mod:`sklearn.neural_network`
 .............................
 
diff --git a/sklearn/neighbors/_regression.py b/sklearn/neighbors/_regression.py
index 1bc02fedba212..c6b9cd90a020d 100644
--- a/sklearn/neighbors/_regression.py
+++ b/sklearn/neighbors/_regression.py
@@ -233,7 +233,7 @@ def predict(self, X):
         if weights is None:
             y_pred = np.mean(_y[neigh_ind], axis=1)
         else:
-            y_pred = np.empty((X.shape[0], _y.shape[1]), dtype=np.float64)
+            y_pred = np.empty((len(X), _y.shape[1]), dtype=np.float64)
             denom = np.sum(weights, axis=1)
 
             for j in range(_y.shape[1]):
diff --git a/sklearn/neighbors/tests/test_neighbors.py b/sklearn/neighbors/tests/test_neighbors.py
index a1e0b01ef3eeb..61aebad5f6598 100644
--- a/sklearn/neighbors/tests/test_neighbors.py
+++ b/sklearn/neighbors/tests/test_neighbors.py
@@ -27,7 +27,10 @@
 from sklearn.metrics.tests.test_dist_metrics import BOOL_METRICS
 from sklearn.model_selection import cross_val_score
 from sklearn.model_selection import train_test_split
-from sklearn.neighbors import VALID_METRICS_SPARSE
+from sklearn.neighbors import (
+    VALID_METRICS_SPARSE,
+    KNeighborsRegressor,
+)
 from sklearn.neighbors._base import (
     _is_sorted_by_data,
     _check_precomputed,
@@ -2038,3 +2041,18 @@ def test_neighbors_distance_metric_deprecation():
 def test_valid_metrics_has_no_duplicate():
     for val in neighbors.VALID_METRICS.values():
         assert len(val) == len(set(val))
+
+
+def test_regressor_predict_on_arraylikes():
+    """Non-regression test for #22687"""
+    x = [5, 6, 4, 3, 4, 3, 1]
+    x = [k for k in zip(x, x)]
+
+    y = [2, 5, 2, 3, 7, 6, 8]
+
+    def _weights(dist):
+        return [[1] * dist.shape[0]]
+
+    est = KNeighborsRegressor(algorithm="brute", weights=_weights)
+    est.fit(x, y)
+    assert_allclose(est.predict([[2, 4]])[0], 20.0)


Code review comments:
diff_hunk: @@ -233,7 +233,7 @@ def predict(self, X):
         if weights is None:
             y_pred = np.mean(_y[neigh_ind], axis=1)
         else:
-            y_pred = np.empty((X.shape[0], _y.shape[1]), dtype=np.float64)
+            y_pred = np.empty((len(X), _y.shape[1]), dtype=np.float64)
path: sklearn/neighbors/_regression.py
text: @user1:
At this point, I think we can use `neigh_dist` to infer the number?

```suggestion
            y_pred = np.empty((neigh_dist.shape[0], _y.shape[1]), dtype=np.float64)
```

@user2:
Otherwise, we would tend to use `_num_samples` for array-like.

@author:
Done!