Deep learning: Getting Started with Feed Forward Neural Networks
By: Evidence Monday
Tags: Feed Forward, Getting Started, Deep Learning, Machine Learning, Neural Networks

What you need:

A good PC (atleast 4 GB RAM) Anaconda. You can download here Good internet of course

Steps

Load datasets Make datasets Iterable Create Model Class (we’ll be using ReLU as our activation function) Instantiate Model Class Instantiate Loss class Instantiate Optimizer class Train Model

Let’s get Started

Step 1: Loading Dataset (MNIST Dataset)

Here we are using the MNIST dataset. True is assigned to the training dataset and False for the Test dataset for obvious reasons; we are only interested in training the train dataset. You can always leave your download as ‘True’, because it basically keeps check of whether you’ve downloaded the dataset or not. So you should have something like this:

Step 2: Making dataset Iterable

This just means that we can iterate through the datase. batch_size means “every iteration, we feed ‘x’ images to our model”. You are free to choose any number you want. n_iters is: “within one epoch, how many iterations are we looking at”. One epoch means going through your whole dataset once. num_epochs: “we are determining the number of epochs based on the number of iterations we have”.

Shuffle = True means; after every epochs we shuffle the image so we do not go through it in a certain manner. Note: “We dont shuffle test dataset because when you are testing your model, you dont have to run multiple epochs”

Step 3: Creating Model Class

The hidden size determines, how many non-linearity dimensions we have. We are using ReLU (Rectified Linear Unit)activation function here, because it gives us the best benefit and it is less computationally expensive compared to tanh and sigmoid activation function.

Input dimension = 748 because the MNIST image is a square of size 28 *28 and when flattened you’ll get 784.

Output dimension is 10; we have numbers 0, 1, 2, 3, 4, 5, 6, 7, 8, 9.

Hidden dimension (Number of neurons, Number of non-linear activation function): 100; it can be any number. The tip is, if you increase the number, you increase the model’s capacity. So it will learn more. The con is, you might need more data as you increase the number and that may not probably be necessary.

Step 4: Instantiating Model Class

What’s happening here is that we are assigning, values to these objects and then feeding them to our model class.

Step 5: Instantiating Loss Class

The loss class calculates the SoftMax Output and does a cross entropy loss.

Tip:

Feedforward Neural Network — Cross Entropy Loss

Logistic Regression — Cross Entropy Loss

Linear Regression — Mean Sqaure Error

Step 6: Instantiate Optimizer Class

We do this because we want to update our model’s parameters with the loss that we have. So on every iterations we have our gradient.

Equation:

Θ = Θ- η . ∆

Θ: Parameters (our variables)

η: Learning rate (how fast we want to learn)

∆: parameter’s gradient

parameters = parameters -leaning rate * parameters_gradients

Remember: After 100 images, we get our loss, gradients, then update our parameters.

So we use SGD (Stochastic Gradient Descent)to update our parameters. We do not want our learning rate to be too high because we simply might miss the best solution, so we want to take small little steps down the road.

Parameters (Briefly)

We can print our model parameters accordingly, we get an output that gives us some sense that our parameter is a generator. So we can access the length by passing through a list, then we find the length. The length is 8, which tells us that we have 8 groups of parameters. So we access each group

Step 7: Train Model Class

To be continued…