Car Image Recognition with Convolutional Neural Network Applications
By: Patricia L Schutter
Tags: Data Science, Image Classification, Transfer Learning

Data Description

The data was obtained from the ‘Cars’ dataset publicly available on Stanford University’s Artificial Intelligence Laboratory website. A few of the images are shown below.

Figure 2

There are 196 models in the Stanford car dataset. However, for the purposes of the classifiers built in this project, cars with multiple variants but virtually identical visual characteristics and similar cars from different model years were put in the same class. For instance, images that belong to Bentley Continental GT Coupe 2012 and Bentley Continental GT Coupe 2007 were put in the same class since the cars look similar. This brought the total number of classes down to 157.

The car_ims.tgz file contains all the images used for this project and the cars_annos.mat file contains the names of the cars corresponding to each image file. Code was written to create folders that were named after the 157 distinct classes for the train and test sets. After the directories were created, the images were copied from the unzipped car_ims.tgz file and put into the respective train and test folders. An 80/20 split was used to split the train and test data; there were 12948 train images and 3237 test images. The train images were further split using an 80/20 split for the purposes of validation.

Image Preprocessing

Since the different models make use of distinct preprocessing techniques, the image preprocessing was done on a model by model basis. For the transfer learning techniques, the images were preprocessed using their specific Keras ‘preprocess_input’ function. It was also made that the images were loaded in using the appropriate image sizes before running the preprocessing functions.

Basic CNN Model

The basic CNN model makes use of five convolutional layers with ‘relu’ activation and the ‘adam’ optimizer. Initially, there was clear evidence of overfitting and in order to reduce the amount of overfitting, dropout layers were added. Regularization in the form of ‘l1’ and ‘l2’ regularizers were also added to help prevent overfitting. In the end, there were 1,818,679 total parameters and all of them were trainable. Below is a snippet of the code:

Figure 3

The model was allowed to train for a 100 epochs with a callback option that monitored the loss. After the model stopped training for 59 epochs, the accuracy and loss curves below were obtained:

Figure 4

For this model, the test accuracy was only 15.6%. This called for transfer learning techniques to be used in the hope that they would yield higher accuracies.

Transfer Learning

Most convolutional neural networks are not trained from scratch; instead, people use models that have already been trained on another dataset and use said models to help classify the data. This method of solving problems, where knowledge gathered from solving problem A is used to solve problem B is called transfer learning. The idea is that the model will still be able to detect edges and other features based on knowledge it gained from solving a previous problem.

For the purpose of this project, five different transfer learning models have been trained and tested:

VGG16 ResNet50 ResNet101 InceptionV3 Xception

VGG-16

The first model we used to upgrade from the basic CNN model was VGG-16. This model has 16 convolutional layers that are each composed of trainable parameters. Although it’s more complex with 138 million parameters, it’s architecture is very uniform. The convolution kernels are of the uniform size 3 x 3, and it has a stride of 1. Figure 5 shows the process: two convolutional layers, followed by pooling, then another two, again followed by pooling, then three sets of three convolutional layers, each followed by pooling, all the way down to the three dense layers. It is then followed by a softmax function for output. The hidden layers all use ReLU and the three dense layers at the end are fully-connected. We used the VGG-16 model library that uses pre-trained weights, but we unfroze the last two layers before fitting on our own data set to allow for more flexibility. We also pre-processed the images for the training and test sets.

Figure 5

Figure 6

Figure 7

Our results with VGG-16 were better than the previous basic CNN model, but still not very impressive. The model had 10 epochs, and the batch size of 50 gave us a total of 259 batches.

We decided on these numbers after some trial and error. The left side of Figure 7 shows the accuracy curve where the test accuracy leveled out after about 10 epochs around an optimal value of .4963. The training accuracy continued to rise (as expected) to .8165 after 10 epochs.

The right side of Figure 7 shows the loss curve, which also levels out at a test loss just above 2, the minimum being 2.26.

An accuracy score of about 50% is unsatisfactory, but VGG-16 does have potential in other circumstances. For instance, with access to proper equipment with higher machine capability, we could pre-train all of the weights rather than a subset. This could potentially help us achieve higher accuracy scores when predicting the car images. VGG-16 has also helped pave the way for more advanced neural networks like ResNet.