Making a Handwritten Digit Recogniser program using nearest neighbour classifier | by Salil Jain | Towards Data Science

Nearest neighbour classifier is one of the most basic algorithm in machine learning. This tutorial describes a very basic method to make a digit recognizer program. We’ll not use any python machine learning library here. So lets get started
from sklearn import datasetsimport matplotlib.pyplot as pltimport numpy as np
sklearn’s digits data will be used for this tutorial.
digits = datasets.load_digits()
digits.data contains image in a linear array (64 x 1 in this case) and digits.images stores them in 8 x 8 array. Let us display a number
fig = plt.figure()plt.imshow(digits.images[23],cmap = plt.cm.gray_r)txt = "This is %d"%digits.target[23]fig.text(0.1,0.1,txt)plt.show()
this will give show us the image at index 23
Now let us see how this same image looks like as an array.
digits.images[23]
This will give output as
array([[  0.,   1.,   8.,  12.,  15.,  14.,   4.,   0.],       [  0.,   3.,  11.,   8.,   8.,  12.,  12.,   0.],       [  0.,   0.,   0.,   0.,   2.,  13.,   7.,   0.],       [  0.,   0.,   0.,   2.,  15.,  12.,   1.,   0.],       [  0.,   0.,   0.,   0.,  13.,   5.,   0.,   0.],       [  0.,   0.,   0.,   0.,   9.,  13.,   0.,   0.],       [  0.,   0.,   7.,   8.,  14.,  15.,   0.,   0.],       [  0.,   0.,  14.,  15.,  11.,   2.,   0.,   0.]])
The more darker the pixel, greater is the number. If you move your cursor over greater numbers on the array you can see that it is making a 3.
We will choose first 100 images as our training data set.
x = 100 #length of training data setX_train = digits.data[0:x]Y_train = digits.target[0:x]
Testing data set
pred = 813X_test = digits.data[pred]print "X_test's real value is %d"%digits.target[pred]
For our program we will find the distances of test image with each training image and the one with least distance is the predicted value. What I mean by distance is the Euclidean distance in 64 dimensions. To calculate the Euclidean distance we will simply find the difference between the numbers at each index, square them, add them and then take square root. So, lets define dist function.
def dist(x,y): return np.sqrt(np.sum((x-y)**2))
This function will work for any dimensional array as long as both x and y have same dimensions.(One of the reasons why I love python :P).
Now let us predict the test data using the dist function.
l = len(X_train)distance = np.zeros(l) for i in range(l): distance[i] = dist(X_train[i],X_test)min_index = np.argmin(distance)print "Preditcted value is ",print(Y_train[min_index])
This will print the predicted value which is 9 in this case.
Now, let us find how well our model predicts.
l = len(X_train)no_err = 0distance = np.zeros(l)for j in range(1697,1797): X_test = digits.data[j] for i in range(l):  distance[i] = dist(X_train[i],X_test) min_index = np.argmin(distance) if Y_train[min_index] != digits.target[j]:  no_err+=1print "Total errors for train length = %d is %d"%(x,no_err)
Our testing data set has 100 examples. In the for loop it predicts the number for image at j index and compares it with its actual value and then prints the total number of errors. When x = 100 14/100 values are wrongly predicted and for x = 1696 2/100 values are wrongly predicted. So our model predicts images with 98% accuracy.
This is part 2 of the tutorial where we will do prediction on custom inputs. You can use any Image editor here we will use G.I.M.P.
Follow these steps
Open GIMPCtrl + NIn the next window put height and width both as 8. You may need to zoom in as the sheet is very smallNow scroll. If the pointer’s size is changing then go to next step, otherwise follow this so that you can change the pointer size.Decrease the pointer’s size to minimum.Go to Tools → Paint Tools → Paintbrush or Press P.Tell the person to draw a number in that file.Go to File → Export and save you image as test.png.
Open GIMP
Ctrl + N
In the next window put height and width both as 8. You may need to zoom in as the sheet is very small
Now scroll. If the pointer’s size is changing then go to next step, otherwise follow this so that you can change the pointer size.
Decrease the pointer’s size to minimum.
Go to Tools → Paint Tools → Paintbrush or Press P.
Tell the person to draw a number in that file.
Go to File → Export and save you image as test.png.
Python code to access the image test.png
from scipy import miscimage = misc.imread("/path_to_test.png/test.png",flatten = 1)
Now if you print digits.images[0] you will see that white pixel is equivalent to 0 and more the value, more darker is the pixel but if you print image which is in the standard form, 255 is for white and 0 for full black. So we have to convert it so that white corresponds to 0 and then we will reshape image from 8 x 8 to 64 x 1
image1 = 255 - imageimage2 = np.reshape(image1,64)
If you look at any example of digits.images you will see that the most black pixel corresponds to ~15 but the maximum value of any pixel in your custom input can be 255 so to avoid any errors due to this effect we will normalise the data(i.e. re scale it in between 0 and 1). For that we will just find the maximum number of the array and divide every element by that maximum number.
# Training datafor i in xrange(len(X_train)): maximum = np.amax(X_train[i]) X_train[i]/=maximum#Testing datamaximum = np.amax(X_test)X_test/=maximum
Now we are good to go. Let’s predict :)
l = len(X_train)distance = np.zeros(l) #This will store the distance of test from every training valuefor i in range(l): distance[i] = dist(X_train[i],X_test)min_index = np.argmin(distance)# print X_testprint "Preditcted value is "print(Y_train[min_index])
This will print
Predicted value is 2
You can make more images and predict them.
All the code is available here[https://github.com/Salil-Jain/DigitRecognizer]