Build your perceptron neural net from scratch
Tags: ai, machinelearning, deeplearning, neuralnetworks
Published: 2017-10-10 17:27:33

---
title: Build your perceptron neural net from scratch
published: true
description: Code your own perceptron from scratch!
tags: Ai, machine learning, deep learning, neural networks.
cover_image: https://cdn-images-1.medium.com/max/1200/1*qJGkvQB_okmAVeX3k68D_A.jpeg
---

_This was originally published on [Medium](https://medium.com/@ismailghallou/build-your-perceptron-neural-net-from-scratch-e12b7be9d1ef.)_

Letâ€™s start by defining the terms first, Ai (Artificial intelligence), ANN (Artificial neural networks), Machine learning & Deep learning.

The field of **AI research** defines itself as an area of computer science that deals with giving machines the ability to seem like they are intelligent, which we havenâ€™t yet reached since neuroscience havenâ€™t solved intelligence yet.

**ANN or a neural net** is a computing system inspired by the biological neural networks that constitute biological brains but they look so different.

![ON the left: The biological neuron graph & on the right: the artificial neural network](https://cdn-images-1.medium.com/max/1200/1*qJGkvQB_okmAVeX3k68D_A.jpeg)

**Machine learning** is a field within Ai that focuses on the design of algorithms that can learn from a given data and results which we call â€œtraining dataâ€ to make a prediction based on that given data from new input data.

**Deep learning** is a sub-field within Machine learning which focuses on the same goal but which uses neural networks & deep neural networks.

![Picture taken from Nvidia](https://cdn-images-1.medium.com/max/1200/1*vdHyRQfjt9__LKGgJsC2Kg.png)

So whatâ€™s the perceptron then ? Well the perceptron is an algorithm for supervised learning (which means we know the result weâ€™re trying to get, say like feeding the size & location of a house to predict the price, in the other hand there is unsupervised learning which is used to draw results from datasets consisting of input data without labeled responses).

The perceptron receives input data multiplied by random weights and adds a bias value, put in an activation function to get a result, if the result value is wrong, it uses back propagation & gradient descent to go back & tweak the weights to get a correct result.

![It can be more than 2 inputs !](https://cdn-images-1.medium.com/max/1000/1*obgiEBA8R8Bse2CW9_WN6g.jpeg)

I know there are some terms there you didnâ€™t get, so letâ€™s go ahead and explain that slowly, letâ€™s say youâ€™re a farmer and you want to classify two types of flowers manually, the best way to do this manually is to take the length & width of each flowersâ€™ paddle and represent them on a graph just like shown bellow:

![Data was taken from Kaggle.com](https://cdn-images-1.medium.com/max/800/1*17HpgWaZYbJTsCKfw7WyAQ.jpeg)

Noticed the unknown flower weâ€™re trying to figure its type, as you can see on the graph itâ€™s represented among the type 2 of flowers which means it is a type 2 flower.

What youâ€™ve just did manually is called linear regression, youâ€™ve represented as many flowers you have on the graph as a training data, then you noticed that pattern and drew a line between the two types.

![pattern](https://cdn-images-1.medium.com/max/800/1*3nUzKdMe3HJ-8VsD7plCtw.jpeg)

So now letâ€™s automate it using our own perceptron built from scratch, the activation function weâ€™ll be using is Sigmoid, hereâ€™s how it looks like on the graph:

![Sigmoid function](https://cdn-images-1.medium.com/max/1200/1*_ek5I9jLpMREpmPygtN0KA.jpeg)

Now letâ€™s understand how back propagation & gradient descent work, and to do so we will need:

**The cost function** or also known as squared error function also called cost function:

![Cost, loss or squared error function](https://cdn-images-1.medium.com/max/800/1*yjUSvPUQmsXiF1QaAK3i1Q.jpeg)

**The derivative of the sigmoid function**

![Derivative of the sigmoid function](https://cdn-images-1.medium.com/max/800/1*Hq37bJswK6WICT1vuEVE2g.jpeg)


**The derivative of the cost function**

![Derivative of the cost function](https://cdn-images-1.medium.com/max/800/1*Vwm6Qp0WiWgXZ-D5jJyFIA.jpeg)

**The slope of the cost function** is: the derivative of the cost function multiplied by the derivative of sigmoid.
**Learning rate**: we will go with 0.2
**For the derivatives of the weights**, itâ€™s the inputsâ€™ values,â€Šâ€”â€Šbecause the derivative of a constant multiplied by a non constant is the constantâ€Šâ€”â€Š for **the derivative of the Bias** itâ€™s 1 because the Bias is a non constant multiplied by 1.
Then weâ€™ll be able to get **the new weights & bias** values like shown bellow:

![& weâ€™re done :D](https://cdn-images-1.medium.com/max/800/1*zAbS_BSLKUktsc6KyPwzTw.jpeg)

{% gist 
https://gist.github.com/smakosh/cf3d9459c4f39f29def556471472aab4 %}

Check the code on [Github](https://github.com/smakosh/Perceptron-neural-net-from-scratch/blob/master/Linear_regression_perceptron.ipynb) & on [Kaggle](https://www.kaggle.com/smakosh/perceptron-neural-net-from-scratch/notebook) .