scikit-learn/scikit-learn — pull request #3760 by Garrett-R (2014-10-12 09:43:28+00:00)
Title: Fix KNeighborsRegressor and RadiusNeighborsRegressor returning NaN predi...

A `KNeighborsRegressor` with distance weighting will predict `NaN` if the data to predict on happens to match a data point from the training set. (This actually happened to me with some data from a Kaggle competition)

For example,

from sklearn import neighbors
import numpy as np

X = np.array([[1.0],[2.0],[3.0]])
y = np.array([1.0, 2.0, 3.0])

clf = neighbors.KNeighborsRegressor(n_neighbors=3, weights='distance')

clf.fit(X,y)
print(clf.predict([[1.0]]))

The output is `NaN`. This seems like undesirable behavior. I would have expected `pred` to be `1.0`. In addition, `RadiusNeighborsRegressor` suffers from this problem.

Note that the infinities are already handled properly by `KNeighborsClassifier` and `RadiusNeighborsClassifiers`. For example,

from sklearn import neighbors
import numpy as np

X = np.array([[1.0],[2.0],[3.0]])
y = np.array([1, 0, 1])

clf = neighbors.KNeighborsClassifier(n_neighbors=3, weights='distance')

clf.fit(X,y)
print(clf.predict([[1.0]]))

This outputs `1` as expected.

This pull request is my proposed solution. Note that if there are two points with distance=0, it chooses one of them, which is consistent with the warning given in the [KNeighborsRegressor docs](http://scikit-learn.org/dev/modules/generated/sklearn.neighbors.KNeighborsRegressor.html).

This is my first pull request for any project, so let me know if I'm doing something wrong!