Dann.js - Creating a neural network which learns and counts!
Tags: dannjs, neuralnetworks, javascript, machinelearning
Published: 2021-08-12 07:21:26

Ever wanted to learn how huge neural networks work, and **create** one? Dann.js makes is as easy as ever!

Dann.js is a neural network library for JavaScript, which makes creating neural networks super simplified. It acts as a playground for experimentation with deep neural networks. It is also fast to implement, which makes it a great choice for various applications.

In this tutorial, we will learn how to set up Dann.js and teach a neural network how to count!

### What you will need

* Node.js
* A computer with more than 2GB of ram and a good CPU

### Getting started
#### Setup
To get Dann.js setup, we first install it as a NPM package.

So to install Dann.js, we do
```
npm i dannjs
```
This installs Dann.js as a package for Nodejs, so we can use it in our JavaScript.
#### Let's get started!
So open up your favorite editor (I personally recommend Atom), and create a new project. In the main JavaScript file, which is usually `main.js` or `index.js` (If there isn't one create it), and import Dann.js using `require` statements as follows:
```js
const Dannjs = require('dannjs');
const Dann = Dannjs.dann;
```
This imports the `dannjs` module we installed into the constant `Dannjs` so we can refer to it later. This also initializes the `Dann`, the basic class of the Neural Network.
#### Creating the network
Now we have the Dannjs module imported, we should create our own neural network using instances of the `Dann` class.

Since in this project we are going to make a network which counts numbers in binary, we are going to create a basic network called `CountingDan`. You can name it whatever you want.

The basic syntax for creating a Neural Network instance is `Dann(inputneurons,outputneurons)`. To start basic, we are giving our neural network 4 input neurons and 4 output neurons.
```js
const CountingDan = Dann(4,4);
```
I've tracked down that 16 (4*4) neurons functioned good enough. You could try different things with this worth later.
#### Finishing the creation
We're not done at this point! We need to add some 'hidden' layers.  A hidden layer is essentially a neuron layer that can perform calculations. The name 'hidden' comes from the way that you don't have to see the values of every neuron, in contrast to the input/output layers. You can learn more about hidden layers & the basics surrounding it here. We are also going to set the activation function to `leakyReLU`

```js
countingDan.addHiddenLayer(16,'leakyReLU');
countingDan.lr = 0.01;
```
#### Finishing the creation of the counting network
Technically, we have finished the creation of the network `countingDan`. You can still experiment more, but this should be enough for our current project.

We should test out model by using the `.log()` method, which essentially displays information about our network, and by feeding the network some data to process:
```js
countingDan.log();
countingDan.feedForward([0,0,1,0],{log:true});
```
The `.feedForward()` method takes a array, which it feeds to the input neurons of the network. Remember, we specified 4 input neurons for our network? So we are passing 4 binary bits in an array, one for each neuron. The log parameter specifies that it should tell every processing and log the output.

In my case, it outputted this:
```
Dann NeuralNetwork:
  Layers:
    Input Layer:   4
    hidden Layer: 16  (leakyReLU)
    output Layer: 4  (sigmoid)
  Other Values:
    Learning rate: 0.01
    Loss Function: mse
    Latest Loss: 0

Prediction:  [0.5398676080698,0.6730957170697,0.6748749672290,0.6377636387674]
```
It can be different in your case, as we never trained the model and it just gives some random results, as you can expect from a newborn baby!
### Training the model
#### Setting up the dataset
To train the model, we’re going to need a dataset. Here is a lightweight JavaScript dataset for 4-bits binary counting. It basically looks like this:

```js
const dataset4bit = [

    //...
    {
        input:[1,0,1,0],
        target:[1,0,1,1]
    },
    //  {
    //      input:[1,0,1,1],
    //      target:[1,1,0,0]
    //  },
    {
        input:[1,1,0,0],
        target:[1,1,0,1]
    },

    //...
];
```
You can download the dataset from [this link] (https://gist.githubusercontent.com/matiasvlevi/6ef03e4fe0ca92cb466c6d19a7cfce6a/raw/92cf6467438d5d1689299cfb0264c0b2ba526220/4bitCountingData)

We can see that this dataset contains one number x in 4-bit binary, as the input value and the number x+1 in 4-bit binary as the target value. I commented out the element [1,0,1,1] so we can have a test sample the neural network has never seen. To access the data, we can copy the code included in the GitHub gist above and save it in binaryDataset.js in the same directory as our project. We can then require the file as a module:
```js
const dataset = require('./binaryDataset.js').dataset;
```
Here we are importing the object `dataset` from the file `binaryDataset.js`.

We can now simply access the object data as:
```js
dataset[i].input
dataset[i].target
```
as attributes.

#### Training the model
Now that we have access to the dataset let’s apply it by calling `.backpropagate()` method for each data point in our dataset array. This will tune the weights of the model according to the data you give it.

```js
for (data of dataset4bit) {
    countingDan.backpropagate(data.input,data.target);
}
```
As we have the main dataset object defined as `dataset4bit`, we access it that way, and train the model for every bit of the dataset by calling `.backpropagate()` once for every bit.

Sadly, one epoch (One training pass) is not enough for any dataset to completely train, as you would expect from a child.

So we should train it multiple times, say `100000` times?
This will allow the network to train from every bit:
```js
const epoch = 100000;
for (let e=0; e < epoch;e++) {
    for (data of dataset) {
        countingDan.backpropagate(data.input,data.target);
    }
}
```
### Running the network
Now we have trained it enough, we should run the model!
```js
countingNN.feedForward([1,0,1,1],{log:true});
```
This outputs:
```
Prediction:  [0.999884854,0.9699951248,0.020084607062,0.008207215405]
```
Which seems good enough, as it is pretty close to `[1,1,0,0]`, which we wanted as an answer.

### Finishing up
You can experiment with many datasets, and change the values as you like. The whole code used in this tutorial is:

```js
const Dannjs = require('dannjs');
const Dann = Dannjs.dann;
const dataset = require('./binaryDataset.js').dataset;

const countingDan = new Dann(4,4);
countingDan.addHiddenLayer(16,'leakyReLU');

countingDan.lr = 0.01;

countingDan.feedForward([1,0,1,1],{log:true});
const epoch = 100000;
for (let e=0; e < epoch;e++) {
    for (data of dataset) {
        countingDan.backpropagate(data.input,data.target);
    }
}
countingDan.feedForward([1,0,1,1],{log:true});
```