Machine Learning Perceptron Implementation | by Tarun Gupta | Towards Data Science

In this post, we are going to have a look at a program written in Python3 using numpy. We will discuss the basics of what a perceptron is, what is the delta rule and how to use it to converge the learning of the perceptron.
The perceptron is an algorithm for supervised learning of binary classifiers (let’s assumer {1, 0}). We have a linear combination of weight vector and the input data vector that is passed through an activation function and then compared to a threshold value. If the linear combination is greater than the threshold, we predict the class as 1 otherwise 0. Mathematically,
Perceptrons only represent linearly separable problems. They fail to converge if the training examples are not linearly separable. This brings into picture the delta rule.
The delta rule converges towards a best-fit approximation of the target concept. The key idea is to use gradient descent to search the hypothesis space of all possible weight vectors.
Note: This provides the basis for “Backpropogation” algorithm.
Now, let’s discuss the problem at hand. The program will read a dataset(tab separated file) and treat the first column as the target concept. The values present in the target concept are A and B, we will consider A as +ve class or 1 and B as -ve class or 0. The program implements the perceptron training rule in batch mode with a constant learning rate and an annealing(decreasing as the number of iterations increase) learning rate, starting with learning rate as 1.
where Y(x, w) is the set of samples which are misclassified. We will use the count or the number of misclassified points as our error rate(i.e. | Y(x, w)|). The output will also be a tab separated(tsv) file containing the error for each iteration, i.e. it will have 100 columns. Also, it will have 2 rows, one for normal learning rate and one for annealing learning rate.
Now, the understanding of what perceptron is, what delta rule and how we are going to use it. Let’s get started with Python3 implementation.
In the program, we are providing two inputs from the command line. They are:
1. data — The location of the data file.
2. output — Where to write the tsv solution to
Therefore, the program should be able to start like this:
python3 perceptron.py --data data.tsv --output solution.tsv
The program consists of 8 parts and we are going to have a look at them one at a time.
import argparse # to read inputs from command lineimport csv # to read and process datasetimport numpy as np # to perform mathematical functions
# initialise argument parser and read arguments from command line with the respective flags and then call the main() functionif __name__ == '__main__':        parser = argparse.ArgumentParser()        parser.add_argument("-d", "--data", help="Data File")            parser.add_argument("-o", "--output", help="output")        main()
The flow of the main() function is as follows:
Save respective command line inputs into variablesSet starting learningRate = 1Read the dataset using csv and delimiter='\t', store independent variables in X and dependent variable in Y. We are adding 1.0 as bias to our independent dataThe independent and dependent data is converted to floatThe weight vector is initialised with zeroes with same dimensions as XThe normalError and annealError are calculated by calling their respective methodsFinally, the output is saved into a tsv file
Save respective command line inputs into variables
Set starting learningRate = 1
Read the dataset using csv and delimiter='\t', store independent variables in X and dependent variable in Y. We are adding 1.0 as bias to our independent data
The independent and dependent data is converted to float
The weight vector is initialised with zeroes with same dimensions as X
The normalError and annealError are calculated by calling their respective methods
Finally, the output is saved into a tsv file
The flow of calculateNormalBatchLearning() is as follows:
Initialisation of a variable e to store the error countA loop is run for 100 iterationsPredicted value is computed based on the perceptron rule described earlier using calculatePredicatedValue() methodError count is calculated using the calculateError() methodWeights are updated based on the equation above using calculateGradient() method
Initialisation of a variable e to store the error count
A loop is run for 100 iterations
Predicted value is computed based on the perceptron rule described earlier using calculatePredicatedValue() method
Error count is calculated using the calculateError() method
Weights are updated based on the equation above using calculateGradient() method
The flow of calculateNormalBatchLearning() is as follows:
Initialisation of a variable e to store the error countA loop is run for 100 iterationsPredicted value is computed based on the perceptron rule described earlier using calculatePredicatedValue() methodError count is calculated using the calculateError() methodLearning rate is divided by the number of the iterationWeights are updated based on the equation above using calculateGradient() method
Initialisation of a variable e to store the error count
A loop is run for 100 iterations
Predicted value is computed based on the perceptron rule described earlier using calculatePredicatedValue() method
Error count is calculated using the calculateError() method
Learning rate is divided by the number of the iteration
Weights are updated based on the equation above using calculateGradient() method
As described in the perceptron image, if the linear combination of W and X is greater than 0, then we predict the class as 1 otherwise 0.
We count the number of instances where the predicted value and the true value do not match and this becomes our error count.
This method is translation of the weight update formula mentioned above.
I am giving away a free eBook on Consistency. Get your free eBook here.
Now, that the whole code is out there. Let’s have a look at the execution of the program.
Here is how the output looks like:
The final program
If you enjoy reading stories like these and want to support me as a writer, consider signing up to become a Medium member. It’s $5 a month, giving you unlimited access to stories on Medium. If you sign up using my link, I’ll earn a small commission at no extra cost to you.
tarun-gupta.medium.com
Here is an index of my stories: