Beginner’s guide to building Convolutional Neural Networks using TensorFlow’s Keras API in Python | by Dr. Varshita Sher | Towards Data Science

Welcome to Part 2 of the Neural Network series! In Part 1, we worked our way through an Artificial Neural Network (ANNs) using the Keras API. We talked about Sequential network architecture, activation functions, hidden layers, neurons, etc. and finally wrapped it all up in an end-to-end example that predicted whether loan application would be approved or rejected.
In this tutorial, we will be learning how to create a Convolutional Neural Network (CNN) using the Keras API. To make it more intuitive, I will be explaining what each layer of this network does and provide tips and tricks to ease your deep learning journey. Our aim in this tutorial is to build a basic CNN that can classify images of chest Xrays and establish if it is normal or has pneumonia. Given the Covid-19 pandemic, I think this would make for an interesting project even for your data science interviews!
Let’s get started!
CNNs work best when the data can be represented in a spatial manner, say an image in MxN pixels. If you data is just as useful after shuffling any of your columns with each other then you cannot use CNN.
For instance, if you recall the loan application dataset from Part 1, it had two columns (or features), namely age and area , and if I were to swap the two columns (before feeding it to my network) it would make no difference whatsoever to my dataset. Hence, ANNs are preferred for such datasets. On the contrary, if I were to swap the columns (which are essentially pixel arrays) in my image, I am surely going to mess up my actual image. Hence, using ANNs is a big no-no and you must use CNNs.
We begin by installing Keras onto our machine. As I discussed in Part 1, Keras is integrated within TensorFlow, so all you have to do is pip install tensorflow in your terminal (for Mac OS) to access Keras in your Jupyter notebook. To check the version of Tensorflow, use tf.__version__.
import tensorflow as tffrom tensorflow import kerasfrom tensorflow.keras.models import Sequentialfrom tensorflow.keras.layers import Activation, Dense, Flatten, BatchNormalization, Conv2D, MaxPool2Dfrom tensorflow.keras.optimizers import Adamfrom tensorflow.keras.metrics import categorical_crossentropyfrom sklearn.metrics import confusion_matrixfrom tensorflow.keras.preprocessing.image import ImageDataGeneratorimport numpy as npimport itertoolsimport osimport randomimport matplotlib.pyplot as plt%matplotlib inline
We will be working with the Chest X-Ray Images Kaggle dataset. Once you download and unzip it, you will see the chest_xray folder that contains images organized neatly into train, valid, and test subfolders. Each of these subfolders contains two sub-sub folders, one for normal chest Xrays and another for Xrays indicatingpneumonia.
Note: Make sure the chest_xray folder is sitting in the same directory as your Python Jupyter notebook.
Setting the paths to the train, validate, and test folders:
train_path = "chest_xray/train"test_path = "chest_xray/test"valid_path = "chest_xray/val"
Note: If you collected your own dataset & it is not neatly organized into the train, valid and test folders, worry not! Just make sure (a) you have saved all the images (normal and pneumonia) in a common folder called chest_xrays AND (b) the name of the image contains either of the two words — normal or pneumonia. From thereon, to organize images into train, valid, and test set, use the following code snippet.
# Organize data into train, valid, test directoriesos.chdir('data/chest_xrays')if os.path.isdir('train/normal') is False:    os.makedirs('train/normal')    os.makedirs('train/pneumonia')    os.makedirs('valid/normal')    os.makedirs('valid/pneumonia')    os.makedirs('test/normal')    os.makedirs('test/pneumonia')    for i in random.sample(glob.glob('pneumonia*'), 500):        shutil.move(i, 'train/pneumonia')          for i in random.sample(glob.glob('normal*'), 500):        shutil.move(i, 'train/normal')    for i in random.sample(glob.glob('pneumonia*'), 100):        shutil.move(i, 'valid/pneumonia')            for i in random.sample(glob.glob('normal*'), 100):        shutil.move(i, 'valid/normal')    for i in random.sample(glob.glob('pneumonia*'), 50):        shutil.move(i, 'test/pneumonia')          for i in random.sample(glob.glob('normal*'), 50):        shutil.move(i, 'test/normal')os.chdir('../../')
We need to put the data into a format that the model expects. So we must put our images into a format of a Keras ImageGenerator . In short, we are going to create batches from train, test, and valid directories using theflow_from_directory()function in the ImageGeneratorclass. And it is precisely these batches of data that will be passed to the sequential model using the fit function. (Previously, we were passing NumPy arrays to the fit function when building ANNs in Part 1, but now we need to pass the batches to the fit function).
As part of the preprocessing (before the batches are created), we will be applying preprocessing_function = tf.keras.applications.vgg16.preprocess_input. This will process our images in the same manner that images get processed when they are passed to the vgg16 model (the CNN model that won the ImageNet competition in 2014).
# Creating train, test and valid batches from the respective directoriestrain_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.vgg16.preprocess_input).flow_from_directory(directory=train_path, target_size=(224,224), classes=['pneumonia', 'normal'], batch_size=10)valid_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.vgg16.preprocess_input).flow_from_directory(directory=valid_path, target_size=(224,224), classes=['pneumonia', 'normal'], batch_size=10)test_batches = ImageDataGenerator(preprocessing_function=tf.keras.applications.vgg16.preprocess_input).flow_from_directory(directory=test_path, target_size=(224,224), classes=['pneumonia', 'normal'], batch_size=10, shuffle=False)
Thetarget_size specifies what height and width we want the images to be resized to. This is important since we might have images of different sizes in our dataset.
The batch_size is set to 10 and the choice of 10 is arbitrary. Another common batch size value used in 32 but ideally, the most optimal value should be found by hyperparameter tuning (we will learn how to do so in the next tutorial).
Batch size specifies how many training samples should be passed to the neural network in one go before the weights of the neural network are updated.
We also specify shuffle = False only for the test set because later on when we test our model, we want to have access to unshuffled labels to plot our confusion matrix. By default, shuffle is set to True.
Note: If you see Found 0 images beloning to 2 classeswhen you run the code above, chances are you are pointing to the wrong directory! Fix that and it should work fine!
Using the plotting helper function from TensorFlow’s documentation.
# plot images in the form of a 1 by 10 grid and resize img to 20x20def plotImages(images_arr):    fig, axes = plt.subplots(1, 10, figsize=(20,20))    axes = axes.flatten()    for img, ax in zip( images_arr, axes):        ax.imshow(img.astype(np.uint8))        ax.axis('off')    plt.tight_layout()    plt.show()
Newt, we will use the Python function next() to grab a single batch of images and corresponding labels from the trainset. Since we set batch_size = 10, we will see we have 10 images in imgs and 10 corresponding labels in labels.
imgs, labels = next(train_batches)plotImages(imgs)print(labels)
Note: imshow expects inputs in the range of [0, 1] or [0, 255] so if the input array does not have this range, it will clip input. Thus, we explicitly cast the images to np.int8 format when displaying.
Note: The images might look a little bit different than the ones in your directories (perhaps a little distorted) but that's because we have performed some vgg16 preprocessing steps on it.
The labels are one-hot encoded vectors that indicate whether the image is pneumonia or normal. A [1 0] represents pneumonia whereas [0 1]means normal Xray.
Recall we introduced a sequential model when we built an artificial neural network in Part 1 for predicting loan application outcomes. In short, the sequential API allows you to create models layer-by-layer.
model = Sequential([    Conv2D(filters=32, kernel_size=(3, 3), activation='relu', padding = 'same', input_shape=(224,224,3)),    MaxPool2D(pool_size=(2, 2), strides=2),    Conv2D(filters=64, kernel_size=(3, 3), activation='relu', padding = 'same'),    MaxPool2D(pool_size=(2, 2), strides=2),    Flatten(),    Dense(units=2, activation='softmax')])
The first hidden layer is a standard convolutional layer, Conv2D, with 32 filters each with a kernel size of 3x3. Remember each filter is dedicated to identifying some aspect of the image, say filter1 detects vertical boundaries of the ribcage, filter2 detects diagonal boundaries of the ribcage, and so on... The patterns that the filters are encoding will become complex over time as we go deeper into the convolutional layers of the network.
Note: The choice of 32 is arbitrary!
A kernel is a small grid of numbers and you pass it over your whole image i.e. all the pixels in your input image, and transforming it based on what those numbers are.
By using different numbers in the kernels, you can perform different types of image preprocessing (during image recognition) such as contrast, blur, sharpen, or edge detection. Generally speaking, the kernel is smaller than the image, and 3x3 is a pretty common choice!
When a kernel reaches the edge of the image, it has to make some decision about what to do where there are no pixels present in the input image. For now, we are going forward with zero-padding the edges, achieved using padding = 'same'. Zero padding means providing a cushion of zeroes outside of the image (as seen in the gif image above). This helps to ensure the dimensionality of the input image isn’t reduced after the convolution operation.
Note: Depending on the kernel size and image size, you might need to make this cushion two or three layers deep.
From Part1, you already know how we have to come up with the input_shape parameter based on our own dataset. Basically, it can be thought of as the shape of a single image that is fed to the network, which in our case is 224x224x3 (height = width = 224 and 3 is for the color channel, more precisely the RGB component). You can verify if that's correct by checking the shape of a random element (say the first element) in imgs using imgs[0].shape which gives out the tuple (224,224,3)as the answer.
Finally, for the first hidden layer, we have the relu activation function that (discussed in Part1). To summarize, an activation function ensures values that are passed on lie within a tunable, expected range. The Rectified Linear Unit (or relu) function returns the value provided as input directly, or the value 0.0 if the input is 0.0 or less.
The second hidden layer is a max-pooling layer, MaxPool2D, which is mainly for reducing the dimensionality of the data. The way it is doing that is by picking a window size ( pool_size = 2x2) from the filtered image (from convo/RELU layer) and then replace the 4 pixels in it with the highest value in that window. Next, it takes a stride of 2 steps to the right (or down or up) and replace those 4 pixels with the highest value. And so on until the entire image has been covered.
The third and fourth hidden layers are again Conv2Dand MaxPool2D, respectively.
Note: The second Conv2D layer has filters = 64 , double the number of filters in the first Conv2D layer (which has 32 filters). This increment in functions is a common practice as we go into the later layers of the network.
Finally, we flatten the output from the convolutional layer and pass it to the Dense layer with units = 2(corresponding to output classes pneumoniaand normal) and softmax activation. The softmax function returns the output probabilities for both classes. (For a detailed description of softmax activation, please read here).
This is how the model summary looks like:
model.summary()
model.compile(optimizer=Adam(learning_rate=0.0001), loss='categorical_crossentropy', metrics=['accuracy'])
Before we start training our model with actual data, we must compile the model with certain parameters. Here, we will be using the Adam optimizer . The loss parameter specifies categorical cross-entropy loss should be monitored at each iteration. The metrics parameter indicates we want to judge our model based on the accuracy.
Note: While the following code runs, you can easily go grab a coffee or something. It took me about 5mins per epoch on a CPU, so approx 50 minutes to run through the 10 epochs.
# recall that we set batch_size = 10 during preprocessingbatch_size = 10 model.fit(    x = train_batches,     steps_per_epoch=train_batches.samples // batch_size,     epochs=10,     validation_data=valid_batches,     validation_steps=valid_batches.samples // batch_size,    verbose=2)
If you observe carefully, we haven’t explicitly stated the labels for the training set (as we did for ANNs in Part1 using the y parameter) and the reason is that they are already incorporated within train_batches iterator.
Thesteps_per_epoch is calculated based on your training data. For example, if the number of samples in our train set is 1000 and the batch size is 20, then the steps_per_epoch is calculated by dividing 1000/20 = 50.
More formally,
steps_per_epoch is an integer and corresponds to the total number of steps (batches of samples) to yield from generator before declaring one epoch finished and starting the next epoch. It should typically be equal to ceil(num_samples / batch_size).
Also, since we are passing validation_data = valid_batches, we need to pass validation_steps as well. It is exactly like the steps_per_epoch parameter, but mainly in regard to the validation data (instead of the training data).
This is how the output from model training looks like, with verbose set to the highest level in terms of the output messages:
As you can see, there is definitely some overfitting going on as train set accuracy is 100% and the training set loss is almost negligible but accuracy on the validation set is only 75%. But hey, even with such a basic model architecture and only six hidden layers, this is not bad at all, to begin with!
If you want to aim for even higher accuracy (~92%), use a deeper model architecture like the one described here.
Other awesome ways to reduce overfitting is with Dropouts and Batch Regularization, which we will be covering in our next tutorial.
# making predictionspredictions = model.predict(x = test_batches, verbose=0)
Using the plotting function directly from Scikit Learn documentation:
def plot_confusion_matrix(cm, classes,                          normalize=False,                          title='Confusion matrix',                          cmap=plt.cm.Blues):    """    This function prints and plots the confusion matrix.    Normalization can be applied by setting `normalize=True`.    """    plt.imshow(cm, interpolation='nearest', cmap=cmap)    plt.title(title)    plt.colorbar()    tick_marks = np.arange(len(classes))    plt.xticks(tick_marks, classes, rotation=45)    plt.yticks(tick_marks, classes)if normalize:        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]        print("Normalized confusion matrix")    else:        print('Confusion matrix, without normalization')print(cm)thresh = cm.max() / 2.    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):        plt.text(j, i, cm[i, j],            horizontalalignment="center",            color="white" if cm[i, j] > thresh else "black")plt.tight_layout()    plt.ylabel('True label')    plt.xlabel('Predicted label')
# confusion matrixcm = confusion_matrix(y_true=test_batches.classes, y_pred=np.argmax(predictions, axis=-1))plot_confusion_matrix(cm, classes = ['Pneumonia', 'Normal'])
And there we have it — our very first CNN model for binary image classification in Python. Please refer to Part 1 to learn how to save the model architecture, weights, and state of the optimizer. In the next tutorial, we will be learning how to improve upon the accuracy of both CNNs and ANNs on the test data.
Until then :)