Build your own deep learning image classification model in Keras | by Edward Jones | Towards Data Science

Image classification is a field of artificial intelligence that is gaining in popularity in the latest years. It has various applications: self-driving cars, face recognition, augmented reality,... .
In this article, you will learn how to build a deep learning image classification model that is able to detect which objects are present in an image in 10 steps.
The full code is available in a shared google collab so you can easily code along while reading the article.
After reading this guide, you will know the following things:
How to set up an coding environment made for deep learning models.
How to connect with a popular image classification dataset (Pascal VOC)
How to create a deep learning convolutional neural network using the combination of Keras & Tensorflow
How to efficiently train a deep learning model using a datagenerator
How to evaluate the datagenerator performance
Please check the google collab for the required packages.
Building a deep learning convolutional network from scratch requires huge amounts of computing power. Ordinary laptops are not equipped to handle these requests.
Luckily, Google comes to our rescue! They have developed an online python notebook which gives users free computing power.
You can enable the computing power feature by select the GPU option in the notebook settings.
We will use the Pascal VOC image dataset for our deep learning model.
The Pascal VOC dataset is an image dataset and is a very popular for computer vision competitions on Kaggle.
Use the Wget package to download the dataset. This package fetches the data and downloads it to your current working directory.
import tarfile!wget -nc http://host.robots.ox.ac.uk/pascal/VOC/voc2009/VOCtrainval_11-May-2009.tar
As a last step, you open the tarfile and extract it.
tf = tarfile.open("/content/VOCtrainval_11-May-2009.tar")tf.extractall()
Good job! You have now successfully loaded in and extracted the dataset.
The current data structure is not optimal for building deep learning convolutional models. Therefore, the data needs to be transformed to a more workable format.
The Pascal VOC dataset that you extracted should contain the two following folders:
Annotations: This folder contains all the information about the image labels.
JPEGImages: This folder contains all the raw images
Create two lists:
Filenames: In this list, you save all the filenames of the pictures. Eg. “2208–001068.jpeg”
Classification: In this list, you save all of the classification labels. Eg. “Bicycle, Sofa”
directory_annotations = '/content/VOCdevkit/VOC2009/Annotations'filenames = []classification = []
You iterate over the annotations directory and extract the filename and label and add it to the respective lists.
for xml_file in os.listdir(directory_annotations):    # Save image for classification and their class label    if os.path.isfile(xml_file):    xml_tree = ET.parse(xml_file)    root = xml_tree.getroot()    imgname = root.find('filename').text.strip('.jpg')    labels = []    for obj in root.findall('object'):    label = obj.find('name').text    labels.append(label)    filenames.append(imgname)    classification.append(labels)
In this step, you will have to preprocess the data:
Split the up the filenames and their respective classification in a training and test set.
label_filenames_temp = os.listdir(directory_annotations)filenames = []for lbl in label_filenames_temp:    filenames.append(lbl.split('.')[0])filecount = len(filenames)indexes = []for index in range(filecount):     indexes.append(index)training_indexes = indexes[:int(filecount*0.7)]validation_indexes = indexes[int(filecount*0.7):int(filecount*0.9)]testing_indexes = indexes[int(filecount*0.9):]
The output labels needs to be converted to numeric values since deep learning networks perform better when the input and output variables are numbers.
directory_images = '/content/VOCdevkit/VOC2009/JPEGImages'directory_annotations = '/content/VOCdevkit/VOC2009/Annotations'labelnames = preprocessing.LabelEncoder()labelnames.fit(["aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow", "diningtable", "dog", "horse", "motorbike", "person", "pottedplant", "sheep", "sofa", "train", "tvmonitor"])
The images are resized to the 224,224,3 format. Literature review advises this when building a deep learning network based on the VGG16 model archtiecture. (Simonyan & Zisserman, 2014)
def generate_from_xml(filename):label = np.zeros((20), dtype = 'float32')tree = ET.parse(os.path.join(directory_annotations, filename + ".xml"))raw_image = cv2.imread(os.path.join(directory_images, filename + ".jpg"))res_img = cv2.resize(raw_image, (224,224))     for elems in tree.iter():         if elems.tag == "object":            name = elems.find("name").text            labelnr = labelnames.transform([name])[0]            label[labelnr] = 1return label, res_img
Training a model on huge datasets requires huge amounts of ram memory.
If you are like me and are not in possession of a supercomputer, you have to use a datagenerator.
A datagenerator feeds the data in small batches to the model. This allows us to train our model without having huge amounts of ram memory.
It is best practice to use a datagenerator when using big datasets (opposed to buying more ram memory). The details of a datagenerator are out of scope in this article but feel free to check out the following link if you are interested.
In the next code block, you create a datagenerator class instance and call it twice:
One time for the training set
One time for the validation set
In this step, you will build the architecture of the classification convolutional neural network.
The model architecture is based on the popular VGG-16 architecture. This is a CNN with a total of 13 convolutional layers (cfr. figure 1).
The following code block indicates that you will be building the model in a sequential way. This means that you will layer after layer.
model = Sequential()
You add 2 convolutional layers.In the convolutional layers, multiple filters are applied to the image to extract different features.
Arguments given:
- Input-shape: The image given should be of the shape (224,224,3).
- Filters: The number of filters that the convolutional layer will learn.
- Kernel_size: specifies the width and height of the 2D convolution window.
- Padding: Specifying “same” ensures that the spatial dimensions are the same after the convolution.
- Activation: This is more of a convenience argument. Here, we specify which activation function will be applied after the convolutional layers. We will apply the ReLU activation function. More on this later.
model.add(Conv2D(input_shape=(224,224,3),filters=64,kernel_size=(3,3),padding="same", activation="relu"))model.add(Conv2D(filters=64,kernel_size=(3,3),padding="same", activation="relu"))
Next, you add 1 maxpool layer.
Pooling is used to reduce the dimensionality of images by reducing the number of pixels in the output of the previous convolutional layer.
- Pool_size= 2,2 -> this is the ‘matrix’ that will go over the output and where the maximum value is taken from
- strides= 2,2 -> the increment of how the pool matrix will move along x & y -axis.
model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
We continue to add layers to our deep learning network. The same logic as described above is applied.
model.add(Conv2D(filters=128, kernel_size=(3,3),padding="same", activation="relu"))model.add(Conv2D(filters=128, kernel_size=(3,3),padding="same", activation="relu"))model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))model.add(Conv2D(filters=256, kernel_size=(3,3),padding="same", activation="relu"))model.add(Conv2D(filters=256, kernel_size=(3,3),padding="same", activation="relu"))model.add(Conv2D(filters=256, kernel_size=(3,3),padding="same", activation="relu"))model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))model.add(Conv2D(filters=512, kernel_size=(3,3),padding="same", activation="relu"))model.add(Conv2D(filters=512, kernel_size=(3,3),padding="same", activation="relu"))model.add(Conv2D(filters=512, kernel_size=(3,3),padding="same", activation="relu"))model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))model.add(Conv2D(filters=512, kernel_size=(3,3),padding="same", activation="relu"))model.add(Conv2D(filters=512, kernel_size=(3,3),padding="same", activation="relu"))model.add(Conv2D(filters=512, kernel_size=(3,3),padding="same", activation="relu"))model.add(MaxPool2D(pool_size=(2,2),strides=(2,2)))
Now the convolutional base is created. To be able to generate a prediction, you have to flatten the output of the convolutional base.
model.add(Flatten())
Add the dense layers. The dense layers feeds the output of the convolutional base to its neurons.
Arguments:
- Units: Number of neurons
- Activation function: Relu
The Relu activation function speeds up training since the gradient computation is very simple (0 or 1). This also implies that negative values are not passed or “activated” on to the next layer. This makes that only a certain number of neurons are activated which makes it computationally interesting.
model.add(Dense(units=4096,activation="relu"))model.add(Dense(units=4096,activation="relu"))
You add a sigmoid layer in order to turn the output of the previous layer into a probability distribution. The sigmoid is ideal for multi label classification so that is why we used sigmoid instead of for example a softmax activation.
The probabilities produced by a sigmoid are independent and are not constrained to sum to one. This is crucial in a classification with multiple output labels.
You set the units argument to 20 since we have 20 possible classes.
model.add(Dense(units=20, activation="sigmoid"))
In step 7, you have to compile the model. You use the RMSprop optimier to be able to reach the global minima. You set the learning rate at 0.001.
RMSprop, root mean square prop, is an unpublished optimization algorithm but is very popular by machine learning practitioners. It reduces the fluctuations in the vertical direction while speeding up the learning in the horizontal direction. This causes our model to converge faster to a global minima. The main difference with the regular gradient descent algorithm is how the gradients are calculated. The formula of the calculation of the gradients is shown in the figure below.
You set the binary cross-entropy loss as your loss function. It is recommended to use this loss function for a multi-label classification since each element belonging to a certain class should not be influenced by the decision for another class.
model.compile(optimizer= keras.optimizers.RMSprop(lr=0.001), loss='binary_crossentropy',metrics=['accuracy'])model.summary()
You use the earlystopping method to stop the training once the model performance stops improving on a hold out dataset. In this way, you automatically have the perfect number of epochs while monitoring overfitting.
You give the earlystopping the instructions to seek a minimum for the validation loss.
The earlystopping method only stops training when no further improvement is detected.
However, the last epoch is not necessarily the one with the best performance.
Therefore, you also use the model checkpoint method. This will save the best model observed during the training based on the validation loss.
filepath = "/content/drive/My Drive/MYCNN/CNN1505_v1.h5"earlyStopping = EarlyStopping(monitor='val_loss', verbose=0, mode='min', patience = 4)mcp_save = ModelCheckpoint(filepath, save_best_only=True, monitor='val_loss', mode='min')
Now, you will start training our deep learning neural network. We use the fit generator from keras to load in the data in batches.This is necessary since our entire training set doesn’t fit in our RAM.
You set the following arguments:
Use multiprocessing: Whether to use process-based threading
Workers: Number of threads generating batches in parallel.
history = model.fit_generator(generator=training_generator,validation_data=val_generator,use_multiprocessing=True,workers=6,epochs = 20,callbacks=[earlyStopping, mcp_save])
When our training has finished, you visualize our training and validation results. Two metrics are plotted:
Model accuracy
Model loss
You see that the model quickly converged from a huge training loss in the first epoch to lower numbers. This fast learning rate is due to the nature of the optimizer chosen (RMS prop) which speeds up convergence.The training process picks the model with the lowest validation loss when this metric has not been improved over four epochs.
df = pd.DataFrame(history.history)print(history.history.keys())# summarize history for accuracyplt.plot(history.history['accuracy'])plt.plot(history.history['val_accuracy'])plt.title('model accuracy')plt.ylabel('accuracy')plt.xlabel('epoch')plt.legend(['train', 'test'], loc='upper left')plt.show()# summarize history for lossplt.plot(history.history['loss'])plt.plot(history.history['val_loss'])plt.title('model loss')plt.ylabel('loss')plt.xlabel('epoch')plt.legend(['train', 'test'], loc='upper left')plt.show()
In our final step, we test our model performance on unseen data (test set).