K Nearest Neighbours Explained. In this article I will give a general... | by Vatsal | Towards Data Science

In this article I will give a general overview, implementation, drawbacks and resources associated to the K Nearest Neighbours algorithm. Supervised learning is a subsection of machine learning generally associated to classification and regression based problems. Supervised learning implies that you are training a model using a labelled dataset. K Nearest Neighbours (KNN) falls under the supervised learning umbrella and is one of the core algorithms in machine learning. It’s a highly used, simple yet efficient example of a non-parametric, lazy learner classification algorithm.
Lazy Learner implies that it doesn’t learn a discriminative function from the training data but rather memorizes the training data instead
Non-parametric implies that the algorithm makes no assumptions about the distribution of the data.
The KNN algorithm classifies unclassified data points based on their proximity and similarity to other available data points. The underlying assumption this algorithm makes is that similar data points can be found near one another. It’s commonly used to solve problems in various industries because its ease of use, application to classification and regression problems, and the ease of interpretability of the results it generates.
K represents the number of nearest neighbours. When K = 1, the algorithm is called the nearest neighbour algorithm. This is the simplest scenario where given an unlabelled position X, the algorithm can predict its label by finding the closest labelled point to X and assigning that as the label.
The algorithm works as follows :
1) Choose the number of K and a distance metric used to calculate        the proximity2) Find the K nearest neighbours of the point we want to classify3) Assign the point a label by majority vote
The choice of K is crucial for the model, if chosen incorrectly it can cause the model to be over / under fit. A K value too small will cause noise in the data to have a high influence on the prediction, however a K value too large will make it computationally expensive.
The industry standard for choosing the optimal value of K is by taking the square root of N, where N is the total number of samples. Of course, take this with a grain of salt as it varies from problem to problem.
You can experiment with various values of K and their associated accuracies. Common practices to determine the accuracy of a KNN model is to use confusion matrices, cross validation or F1 scores.
Below, I’ve listed some of the advantages and disadvantages of using the KNN algorithm.
Advantages
Simple & intuitive — The algorithm is very easy to understand and implementMemory based approach — Allows it to immediately adapt to new training dataVariety of distance metrics — There is flexibility from the users side to use a distance metric which is best suited for their application (Euclidean, Minkowski, Manhattan distance etc.)
Simple & intuitive — The algorithm is very easy to understand and implement
Memory based approach — Allows it to immediately adapt to new training data
Variety of distance metrics — There is flexibility from the users side to use a distance metric which is best suited for their application (Euclidean, Minkowski, Manhattan distance etc.)
Disadvantages
Computational complexity — As your training data increases, the speed at which calculations are made rapidly decreasePoor performance on imbalanced data — When majority of the data the model is being trained on represents 1 label then that label will have a high likelihood of being predictedOptimal value of K — If chosen incorrectly, the model will be under or overfitted to the data
Computational complexity — As your training data increases, the speed at which calculations are made rapidly decrease
Poor performance on imbalanced data — When majority of the data the model is being trained on represents 1 label then that label will have a high likelihood of being predicted
Optimal value of K — If chosen incorrectly, the model will be under or overfitted to the data
In summation, this article outlines that kNN is a lazy learner and non parametric algorithm. It works by assigning a label to a unlabelled point based on the proximity of the unlabelled point to all the other nearest labelled points. Its main disadvantages are that it is quite computationally inefficient and its difficult to pick the “correct” value of K. However, the advantages of this algorithm is that it is versatile to different calculations of proximity, its very intuitive and that it’s a memory based approach.
https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html#sklearn.neighbors.KNeighborsClassifier
If you’ve enjoyed this read then check out my other works as well.