Improving accuracy using deep learning for Fashion MNIST
By: Krishna Mohan Chinni
Tags: Batch Normalization, Deep Learning, Fashion Mnist, Lenet, Pytorch

In this post I will go through a process of how we can improve the results for classification task using deep learning.

Deep learning framework

I will use Pytorch library for implementation. We can install it following this link using conda.

Toy Dataset

I have chosen Fashion MNIST as a toy dataset. Using this dataset I will try to improve the results from a baseline. We can get this dataset from here.

From this link download the following

You will get a archive.zip file. From these files we need following files

fashion-mnist_train.csv

fashion-mnist_test.csv

Here is the explanation of the dataset https://github.com/zalandoresearch/fashion-mnist

Here is sample images from the dataset

Fashion MNIST dataset

Base Model

Lets try simple 3-layer neural network as our base model and then try to improve it with new methods in Deep learning.

Here is a simple example of 3-layer network.

3 — layer network Courtesy: http://deeplearning.stanford.edu/tutorial/supervised/MultiLayerNeuralNetworks/

Loading data

Here is the code to load the Fashion MNIST dataset

Coding the Base model

Now lets code a basic 3 layer neural network and train it.

After running this code you will get the accuracy on test set as follows

3 layer neural network accuracy

So our base model has accuracy of 74.5%. The model has 235,146 parameters. Lets try to beat this with improvements.

LeNet

Now lets improve the performance of our base model using a deep learning model called LeNet.Here is the basic architecture of LeNet

LeNet Architecture

Please refer the wiki article for explanation

Here is the LeNet model code

Here there are only 44,426 parameters and the accuracy we get is 87.2%!

Activation unit

Now lets try to improve over the LeNet model. The recent advancement is the new activation unit called ReLU. Lets replace the Tanh with ReLU and see how the accuracy improves.

Here is the code

And we get accuracy of 88.24%. Thats decent improvement!

Batch Normalization

Batch normalization is a technique used to nromalize the input layer before sending it to next layer by re-centring and re-scaling.

Lets add batch normalization see if it improves the results.

And results are

Great! we improved it to 89.6% without any increase in number of parameters.

Make it big!!

Now lets make the network bigger and see if it improves the results.

And results are…

Cool we got it to 91.3% with 85, 918 parameters. Still better than baseline of 235,146 parameters.

Similarly some tweaking of model gets more improvements.

Make the kernel size of layer 2 to 3 from 5. 112,286 parameters and 91.4%

Making kernel size of layer 1 to 3 from 5. 154,270 parameters and 92.5%

Make the out channels of layer 1 to 32 and layer 2 to 64. 306,622 parameters and 94.0%

Thats amazing! we improved the accuracy from 74.5% to 94.0% with cool deep learning models and techniques.

Here is the plot of parameters vs accuracy

Fashion MNIST improvement

There are still more parameters to tune and more models to explore. Please tweak and let me know what you guys can come up with. I am curious!

Bye for now!