K nearest neighbors, it’s purpose and how to use it | by Robert Miller | Towards Data Science

For those of you who don’t know what k nearest neighbors is, it is a predictive model. Whats its purpose, why use it? Those are the questions we will delve into in this post.
In order for this model to work the data must have a notion of distance. In laymens terms, we need to be able to graph our data.
This model will is non-linear and can be used on classes. What I mean by classes is it can help determine if a data point will be part of one class or another. A good example is determining if someone will be a republican or democrat based on their income and age. This model will prove useful because in essence it looks at the neighboring data points to determine what this new data point will fall into.
Now that we know why this is useful, lets give an example with how to do this in python.
This example will be using pedal width, pedal length, sepal length, and sepal width to try to determine the classification of flower
import pandas as pdurl = 'http://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data'col_names = ['sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'species']iris = pd.read_csv(url, header=None, names=col_names)
Firstly, I imported pandas (a python library) and then read in my dataset and set column names. This puts my data into a nice dataframe that can easily be analyzed.
from matplotlib.colors import ListedColormapcmap_bold = ListedColormap(['#FF0000', '#00FF00', '#0000FF']
The chunk of code above imports color graphing ability .
iris["Species_number"] = iris.species.map({'Iris-setosa':0,'Iris-versicolor':1,'Iris-virginica':2 })
I then made a new column mapping each species to a different bin, the bins being 0,1, or 2.
iris.plot(kind = 'scatter', x = "petal_length", y = "petal_width", c ="Species_number" ,colormap= cmap_bold )
This shows pedal width and petal length to determine the species. The species each being a different color.
feature_cols = ['sepal_length', 'sepal_width', 'petal_length', 'petal_width'] X = iris[feature_cols]y = iris.Species_number
This Makes our X the sepal length, sepal width, petal length, petal width. Our Y is the species column with three bins. We are trying to use X features to determine Y
from sklearn.model_selection import train_test_splitX_train, X_test, y_train, y_test = train_test_split(X, y)
For any good model you will want to do a train test split to avoid over fitting your model. What this does is takes your data and splits in to a training set, and then a testing set to test how well your model is. K nearest neighbors (also knows as KNN) is a lazy model, this means is that it does not use the training data points to do any generalization. So we are not going to train a model. Then why train test split you might ask? Because we still need that testing set to test on the 70% to see if our X variables are good predictors.
from sklearn.neighbors import KNeighborsClassifierfrom sklearn import metrics# make an instance of a KNeighborsClassifier objectknn = KNeighborsClassifier(n_neighbors=1)knn.fit(X_train, y_train)
We then import from sklearn.neighbors to be able to use our KNN model. Using KNeighborsClassifier and then the argument inside determines how many nearest neighbors you want your datapoint to look at. There is no rule of thumb for how many neighbors you should look at. The best way to find out how many neighbors to look at is to test a range of neighbors and determine what has the best accuracy.
y_pred_class = knn.predict(X_test)print metrics.accuracy_score(y_test, y_pred_class)
This will let you see the accuracy score, and since we only had 1 nearest neighbor to check, our score ended up being a 1 (also know as 100%), which has potential to be overfit.
def length(X, y):    empty_list = []    for n in range(1,100):        X_train, X_test, y_train, y_test = train_test_split(X, y)        knn = KNeighborsClassifier(n_neighbors=(n))        knn.fit(X_train, y_train)        y_pred_class = knn.predict(X_test)        empty_list.append(metrics.accuracy_score(y_test, y_pred_class))    return empty_list
This function will test 1–100 nearest neighbors and return the accuracy for each. This will help you look for the best number of neighbors to look at for your model
Once you determine the Best neighbor you can use your model on other data points to put them into a class.
Thats how you use KNN, I hope this helped those of you interested in predicting what classes data points will fall into