How to Create a Residual Network in TensorFlow and Keras
By: Ali Pourramezan Fard
Tags: Keras, Deep Learning, Neural Networks, TensorFlow, Machine Learning

train.py

This file contains a class called Train. below is the constructor of the Train class, containing input_shape, output_shape arguments:

def __init__(self, input_shape, output_shape):

"""

:param input_shape: shape of input images, default is 28 * 28 *1

:param output_shape: number of classes. for mnist_ds, default is 10



"""



self.input_shape = input_shape

self.output_shape = output_shape

Plus, we have a method called train_model(), which we use to create dataset and model and then train the model.

Loading dataset:

since the mnist dataset is now available as part of Keras datasets, we can download it using the below command:

(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()

now, we have :

x_train : an array contains 60,000 grayscale 28*28 pixels image. Accordingly, its shape is 6000*28*28. x_train is used as an input dataset.

: an array contains 60,000 grayscale 28*28 pixels image. Accordingly, its shape is 6000*28*28. x_train is used as an input dataset. y_train : an array contains 60,000 integer numbers, taken into account as the labels for the x_train images. Each item of y_train relates to its corresponding index in x_train.

: an array contains 60,000 integer numbers, taken into account as the labels for the x_train images. Each item of y_train relates to its corresponding index in x_train. x_test : an array contains 10,000 grayscale 28*28 pixels image that we use further to test the performance of our model.

: an array contains 10,000 grayscale 28*28 pixels image that we use further to test the performance of our model. y_test: like y_train, this is an array containing 10,000 integer numbers considered as x_test labels.

Now in order to improve the performance of our neural network, we create our validation set from our training set, using its first 5000 items:

x_val = x_train[-5000:]

y_val = y_train[-5000:]

x_train = x_train[:-5000]

y_train = y_train[:-5000]

Now we need to add another dimension to our image arrays, x_train, x_val, and x_test to make them shape as a real image with 3 dimensions, width, height, and the color channel which is 1 here since our images are grayscale:

x_train = x_train.reshape(x_train.shape[0], x_train.shape[1], x_train.shape[2], 1)

x_val = x_val.reshape(x_val.shape[0], x_val.shape[1], x_val.shape[2], 1)

x_test = x_test.reshape(x_test.shape[0], x_test.shape[1], x_test.shape[2], 1)

And then normalizing the input image by multiplying each image our 255:

x_train = x_train/255.0

x_val = x_val/255.0

x_test = x_test/255.0

Finally, we need to convert our labels to categorical labels:

y_train = keras.utils.to_categorical(y_train, self.output_shape)

y_val = keras.utils.to_categorical(y_val, self.output_shape)

y_test = keras.utils.to_categorical(y_test, self.output_shape)

Creating Model:

We define the model in another file called network_model.py. As shown below, we create an object from NetworkModel class and create the model using the sample_res_net_v0 method. We explain the model further in this tutorial.

net_model = NetworkModel()

model_1 = net_model.sample_res_net_v0(input_shape=self.input_shape, output_shape=self.output_shape)

Compiling and Training Model:

We compile the model using below :

model_1.compile(loss=losses.categorical_crossentropy,

optimizer=adam(),

metrics=['accuracy'])

Since our task is a multi-variant classification, we need to use categorical_crossentropy as our loss function. If you are not familiar with different types of loss function in TensorFlow, I strongly recommend you to take a look at here. Plus, we use Adam as our optimizer, and the only metric that we would like to monitor is accuracy.

Finally, we start training the model as below, for 20 epochs and then save the model and weights:

history = model_1.fit(x_train, y_train,

batch_size=50,

epochs=20,

validation_data=(x_val, y_val))

model_1.save()

Visualizing Accuracy and Loss Figures:

After training the model, the history objects contain some valuable information that we can use in order to visualize both accuracy and loss of the model. Then by using the matplotlib library, we generate and save the figures: