Creating of neural network using JavaScript in 7 minutes!
Tags: javascript, webdev, neuralnetworks, machinelearning
Published: 2019-01-12 12:58:53

---
title: Creating of neural network using JavaScript in 7 minutes!
published: true
description: 
tags: javascript,webdev,neuralnetworks,ml
cover_image: https://cdn-images-1.medium.com/max/1200/1*H-25KB7EbSHjv70HXrdl6w.png
---

Hey, what's up 😁 Today, i'm gonna tell you how to build a simple neural network with JavaScript by your own with no *AI frameworks*. Let's go!

For good understanding you need to know these things:
* OOP, JS, ES6;
* basic math;
* basic linear algebra.

# Simple theory
A neural network is a collection of **neurons** with **synapses** connected them. A neuron can be represented as a function that receive some input values and produced some output as a result.

![Simple neuron](https://cdn-images-1.medium.com/max/800/1*UFiJTvJO4eCuTycsLC3TLQ.png)

Every single synapse has its own **weight**. So, the main elements of a neural net are neurons connected into layers in specific way.

Every single neural net has at least an input layer, at least one hidden and an output layer. When each neuron in each layer is connected to all neurons in the next layer then it's called multilayer perceptron (MLP). If neural net has more than one of hidden layer then it's called Deep Neural Network (DNN).

![DNN](https://cdn-images-1.medium.com/max/800/0*UHkKkn4dcN45xbAc.png)

The picture represents DNN of type **6–4–3–1** means 6 neurons in the input layer, 4 in the first hidden, 3 in the second one and 1 in the output layer.

---

# Forward propagation
A neuron can have one or more inputs that can be an outputs of other neurons.

![Synapses](https://cdn-images-1.medium.com/max/800/0*VGvxOUOcJX5hso_0.png)

* X1 and X2 - input data;
* w1, w2 - weights;
* f(x1, x2) - activation function;
* Y - output value.

So, we can describe all the stuff above by mathematical formula:

![Neuron input](https://cdn-images-1.medium.com/max/800/1*dMljfH2anWtmEbSPkZfBXA.png)

The formula describes neuron input value. In this formula: **n - number of inputs, x - input value, w - weight, b - bias** (w e won't use that feature yet, but only one thing you should know about that now - it always equals to 1).

As you can see, we need to multiply each input value by its weight and summarize products. We have sum of the products of multiplying **x** by **w**. The next step is passing the output value ***net*** through activation function. ***The same operation needs to be applied to each neuron in our neural net.***

Finally, you know what the forward propagation is.

---

# Backward propagation (or backpropagation or just backprop)
Backprop is one of the powerful algorithms first introduced in 1970. [[Read more about how it works.]](http://neuralnetworksanddeeplearning.com/chap2.html)

Backprop consists of several steps you need apply to each neuron in your neural net.

* First of all, you need to calculate error of the output layer of neural net.

![error formula](https://cdn-images-1.medium.com/max/800/1*5yo4SajKz1QqXI8GiW0EoA.png) 

**target**  -  true value, **output**  -  real output from neural net.

* Second step is about calculating ***delta error value.***

![delta error](https://cdn-images-1.medium.com/max/800/1*pTQ8wJoGMxfDizlGuCwveg.png)

**f'**  -  derivative of activation function.

* Calculating an error of hidden layer neurons.

![hidden neuron error](https://cdn-images-1.medium.com/max/800/1*hwZatkVm0_B59j8xJmpNIg.png)

**synapse**  -  weight of a neuron that's connected between hidden and output layer.

Then we calculate ***delta*** again, but now for hidden layer neurons.

![hidden delta](https://cdn-images-1.medium.com/max/800/1*t-80Rqf-twKvFhZp7rtETA.png)

**output**  -  output value of a neuron in a hidden layer.

* It's time to update the weights.

![weights update](https://cdn-images-1.medium.com/max/800/1*EaZ-DpsyZGQQQ538Nw85_A.png)

**lrate**  -  learning rate.

Buddies, we just used the simplest backprop algorithm and gradient descent😯. If you wanna dive deeper then watch this video.

{% youtube Ilg3gGewQ5U %}

And that's it. We’re done with all math. Just code it!!!

---

# Practice
So, we’ll create MLP for solving XOR problem (really, man? 😯).

> From the simplest things to the hardest, bro. All in good time.

Input, Output for XOR.

![XOR](https://cdn-images-1.medium.com/max/1600/1*Ic2H-HbN1tOT4q9ARSJvzQ.png)

We’ll use **Node.js** platform and **math.js** library (which is similar to **numpy** in **Python**). Run these commands in your terminal:

```sh
mkdir mlp && cd mlp 
npm init 
npm install babel-cli babel-preset-env mathjs

```

Let’s create a file called `activations.js` which will contain our activation functions definition. In our example we’ll use classical sigmoid function (oldschool, bro).

{% gist https://gist.github.com/liashchynskyi/62e61e3279799a2cb69d5e18e85aaf73 %}

Then let’s create `nn.js` file that contains `NeuralNetwork` class implementation.

{% gist https://gist.github.com/liashchynskyi/7e8639b4401ea07a15e3f1f3a63858d2 %}

It seems that something is missing.. ohh, exactly! we need to add `trainable` ability to our network.

{% gist https://gist.github.com/liashchynskyi/ecdb65767f2b62fcd73284c49d3cdb13 %}

And just add `predict` method for producing result.

{% gist https://gist.github.com/liashchynskyi/f4a0825ab4250c75119ba65c43dd27f2 %}

Finally, let’s create `index.js` file where all the stuff we created above will joined.

{% gist https://gist.github.com/liashchynskyi/8141d3300125c1fed7df28b60c6a2e02 %}

Predictions from our neural net:

![Predictions](https://cdn-images-1.medium.com/max/1600/0*_-56YIqarmZEn6P6.png)

# Conclusions
As you can see, the error of the network is going to zero with each next epoch. But you know what? I’ll tell you a secret — it won’t reach zero, bro. That thing can take very long time to be done. It won’t happens. Never.

Finally, we see results that are very close to input data. The simplest neural net, but it works!

**Source code is available on my [GitHub](https://github.com/liashchynskyi/skynet).**

[Original article](https://codeguida.com/post/1418) posted by me in my native language.