Build a Deep Learning Model with Keras TensorFlow 2.0
By: Ravi Srivastava
Tags: Deep Learning, Data Science, Neural Networks, Machine Learning

To develop a deep learning model, this is very important to understand what is neural network and how It works. In this article we are going to discuss about the fundamental of neural network and how its important in deep learning.

Source Code link — https://github.com/ravi7mca/MLDL_Learning/blob/master/MNIST_Dataset_Keras.ipynb

What is Neural Network?

Neural Network are deep learning models which are made up of layers. Layers is a data processing module which extract useful information from data. Usually neural network are build up with one input layer and one output layer and in between of input and output layer it may contain many hidden layer. This is a building block of any neural network.

In a neural network, the output of one layer is set as input to the next layer in the sequence and each layer are consists of individual interconnected neurons, and these neurons are the active learning unit in neural network.

In the following neural network, we have four Dense layers, which are densely connected (also called fully connected) neural layers. The first layer is an input layer and last layer is an output layer and in between of these two layer, there are two hidden layer so that total number of layers is four, they are fully connected. Following are the basic structure of neural network.

Now, We have a bit idea of neural network, now the question is how we can create a neural network practically to train a model and the answer is Keras. Keras is a deep learning API written in Python, running on top of the machine learning platform TensorFlow. Now the Keras is a part of TensorFlow 2.0.

Keras & TensorFlow 2.0

Keras is the high-level API of TensorFlow 2.0: an approchable, highly-productive interface for solving machine learning problems, with a focus on modern deep learning. It provides essential abstractions and building blocks for developing and shipping machine learning solutions with high iteration velocity.

For the Installation of TensorFlow you can go through this link — https://www.tensorflow.org/install

The problem we are going to solve here is to classify grayscale images of handwritten digits (28 × 28 pixels) into their 10 categories (0 through 9). We’ll use the MNIST dataset. The MNIST dataset comes preloaded in Keras, in the form of a set of four Numpy arrays.

Building a Deep Learning Model

1- First import the required API

from tensorflow import keras

from tensorflow.keras import datasets

2- Load dataset from Keras dataset

(Xtrain, ytrain), (Xtest, ytest) = datasets.mnist.load_data()

Xtrain and ytrain are from training set, these data will use to train a model. Xtest and ytest are from test dataset and it will use to evaluate the performance model.

3- Lets have a look on train and test dataset

The training dataset have 60,000 training images and test dataset have 10,000 test images and each image is 28 × 28 pixels.

4- The deep learning network architecture

from tensorflow.keras import models # 1- Instantiate Model

model = keras.Sequential()

# 2- Specify Shape of First Layer

model.add(layers.Dense(512, activation = ‘relu’, input_shape = (28 * 28,)))

# 3- Add the layers

model.add(layers.Dense(10, activation= ‘softmax’))

In the above code sample we are using ‘Sequential’ Keras building blocks to create a deep learning architecture. Sequential model consists of a simple stack of layers, and so we can’t use to build complex model. Its very easy to create a Sequential model in keras. These are the steps to use Sequential model in keras. 1- Instantiate Model, 2- Specify Shape of First Layer — the shape of the first layer should be equal to the dimensionality of your input data, that is the number of feature in each record of your training set. 3- Add the layers — there are several standard layer are available for e.g. CNN, DNN, RNN. During to add the layer we need to remember two thing, 1- the input_shape in first layer should be same as feature shape, 2- the last layer activation function and output size should be chosen as per your problem. In this problem we need to predict the label 0–9 so total number of output required 10 and we have used ‘softmax’ activation function, which means it will return an array of 10 probability scores (summing to 1). Each score will be the probability that the current digit image belongs to one of our 10 digit classes. Suppose you have a binary classification problem then your last layer should return 1 output and activation function should be ‘sigmoid’.

4- Compile Model — during the compilation of the model, we need to specify the optimizer and loss function. Compilation of model configure the learning parameters of the model.

A loss function — Loss function will use to measure performance of model during the training process, and thus how it will be able to steer itself in the right direction.

An optimizer — The optimizer uses the loss value to update the network’s weights.

Metrics to monitor during training and testing — Here, we’ll only care about accuracy (the fraction of the images that were correctly classified).

The compilation step

# 4- Compile Model

model.compile(optimizer=’rmsprop’, loss=’categorical_crossentropy’, metrics=[‘accuracy’])

5- Train the model- Once the architecture is designed for deep learning now the model is ready for training. We will train the model for number of epochs and feed the data in batches that is the training data. We will use the method model.fit() to train the model.

Before feeding data to model.fit() function for training, we’ll preprocess the data by reshaping it into the shape the network expects and scaling it so that all values are in the [0, 1] interval. Previously, our training images, for instance, were stored in an array of shape (60000, 28, 28) of type uint8 with values in the [0, 255] interval. We transform it into a float32 array of shape (60000, 28 * 28) with values between 0 and 1.

# Preparing the image data

Xtrain = Xtrain.reshape((60000, 28 * 28))

Xtrain = Xtrain.astype(‘float32’) / 255

Xtest = Xtest.reshape((10000, 28 * 28))

Xtest = Xtest.astype(‘float32’) / 255

We also need to perform the categorical encoding on the labels.

# Preparing the labels

ytrain = keras.utils.to_categorical(ytrain)

ytest = keras.utils.to_categorical(ytest)

Now our data is ready to feed in network to train the model.

# 5- Train the model-

model.fit(Xtrain, ytrain, epochs=5, batch_size=128)

once you run the above code you will get the approx 98% accuracy on training data.

Model accuracy on training data

6- Model evaluation with test data — We will use the method model.evaluate() to evaluate the model. For the evaluation, we will use the test data and we will evaluate the performance for model on test data.

Now let’s check that the model performs well on the test set.

# 6- Model evaluation with test data

test_loss, test_acc = model.evaluate(Xtest, ytest)

print(‘test_acc:’, test_acc)

Model accuracy on test data

As you can see the test accuracy is a bit low compare to training accuracy, this is known as overfitting. To deal with this overfitting, we can build a CNN model with Conv2D layers.

I tried to explain you the very basic concepts of deep neural network. In this example, you have build and train a neural network to classify handwritten digits in less than 20 lines of Python code. I hope you enjoyed this learning. Please leave your suggestion on comment section.