Image Classification Keras Tutorial: Kaggle Dog Breed Challenge | by Connor Shorten | Towards Data Science

Identifying dog breeds is an interesting computer vision problem due to fine-scale differences that visually separate dog breeds from one another. This article is designed to be a tutorial for those who are just getting started with Convolutional Neural Networks for Image Classification and want to see how to experiment with network architecture, hyperparameters, data augmentations, and how to deal with loading custom data for test and train.
All code is written in Python and Keras and hosted on Github: https://github.com/CShorten/KaggleDogBreedChallenge/blob/master/DogBreed_BinaryClassification.ipynb
The dataset we are using is from the Dog Breed identification challenge on Kaggle.com. Kaggle competitions are a great way to level up your Machine Learning skills and this tutorial will help you get comfortable with the way image data is formatted on the site.
This challenge listed on Kaggle had 1,286 different teams participating. The overall challenge is to identify dog breeds amongst 120 different classes.
120 classes is a very big multi-output classification problem that comes with all sorts of challenges such as how to encode the class labels. For example, one-hot encoding the labels would require very sparse vectors for each class such as: [0, 0, ...,0, 1, 0,0, ..., 0].
This tutorial randomly selects two classes, Golden Retrievers and Shetland Sheepdogs and focuses on the task of binary classification.
An additional challenge that newcomers to Programming and Data Science might encounter, is the format of this data from Kaggle. Kaggle provides a training directory of images that are labeled by ‘id’ rather than ‘Golden-Retriever-1’, and a CSV file with the mapping of id → dog breed. I have found that python string function .split(‘delimiter’) is my best friend for parsing these CSV files, and I will show you how this works in the tutorial.
The first part of this tutorial will show you how to parse this data and format it to be inputted to a Keras model. We will then focus on a subsection of the problem, Golden Retrievers vs. Shetland Sheepdogs, (chosen arbitrarily).
The second part of this tutorial will show you how to load custom data into Keras and build a Convolutional Neural Network to classify them.
The third part of this tutorial will discuss bias-variance tradeoff and look into different architectures, dropout layers, and data augmentations to achieve a better score on the test set.
from PIL import Image # used for loading imagesimport numpy as npimport os # used for navigating to image pathimport imageio # used for writing imagesnaimg_dict = {} # id: breedf = open("labels.csv", "r")fileContents = f.read()fileContents = fileContents.split('\n')for i in range(len(fileContents)-1):  fileContents[i] = fileContents[i].split(',')  naming_dict[fileContents[i][0]] = fileContents[i][1]
Now we have a python dictionary, naming_dict which contains the mapping from id to breed. Now we need to build a counting dictionary for each breed to assign labels to images such as ‘Golden_Retriever-1’, ‘Golden_Retriever-2’, ..., ‘Golden_Retriever-67’.
breeds = naming_dict.values()breed_set = set(breeds)counting_dict = {}for i in breed_set:  counting_dict[i] = 0
I. We loop through the images which are currently named as ‘id.jpg’
II. strip the .jpg from them
III. Match them with the breed from the naming dictionary.
IV. We will then name them based on how many of this breed we have already counted
V. Finally, increment the count with this new instance.
for img in os.listdir('./raw_data'):  imgName = img.split('.')[0] # converts '0913209.jpg' --> '0913209'  label = naming_dict[str(imgName)]  counting_dict[label] += 1  path = os.path.join('./raw_data', img)  saveName = './labeled_train/' + label + '-' + str(counting_dict[label]) + '.jpg'  image_data = np.array(Image.open(path))  imageio.imwrite(saveName, image_data)
Now all the images in the training directory are formatted as ‘Breed-#.jpg’. Very useful for loading into the CNN and assigning one-hot vector class labels using the image naming.
Note on Train-Test Split: In this tutorial, I have decided to use a train set and test set instead of cross-validation. This is because I am running these CNNs on my CPU and therefore they take about 10–15 minutes to train, thus 5-fold cross validation would take about an hour. I want the focus of this study to be on how the different ways to change your model structure to achieve a better result, and therefore fast iterations are important. Additionally, I have taken a ~2/3–1/3 Train / Test Split, which is a little more testing instances than usual, however, this is not a very big dataset.
def label_img(name):  word_label = name.split('-')[0]  if word_label == 'golden_retriever' : return np.array([1, 0])  elif word_label == 'shetland_sheepdog' : return np.array([0, 1])
This is called one-hot vector encoding and it produces a better result than encoding each label with ‘0’ or ‘1’. In terms of the neural network structure, this means have 2 neurons in the output layer rather than 1, you will see this in the final line on the CNN code below:
Update (4/22/19): This only true in the case of multi-label classification, not binary classification. In binary classification, the output is treated as 0 or 1 and there is only one output neuron, keras will correct this error during compilation.
model.add(Dense(2, activation = 'softmax'))
When we are formatting images to be inputted to a Keras model, we must specify the input dimensions. Many academic datasets like CIFAR-10 or MNIST are all conveniently the same size, (32x32x3 and 28x28x1 respectively). However, in the ImageNet dataset and this dog breed challenge dataset, we have many different sizes of images. First, we will write some code to loop through the images and gather some descriptive statistics on the maximum, mean, and minimum height and width of the dog images.
def get_size_statistics(DIR):  heights = []  widths = []  for img in os.listdir(DIR):     path = os.path.join(DIR, img)    data = np.array(Image.open(path)) #PIL Image library    heights.append(data.shape[0])    widths.append(data.shape[1])  avg_height = sum(heights) / len(heights)  avg_width = sum(widths) / len(widths)  print("Average Height: " + str(avg_height))  print("Max Height: " + str(max(heights)))  print("Min Height: " + str(min(heights)))  print('\n')  print("Average Width: " + str(avg_width))  print("Max Width: " + str(max(widths)))  print("Min Width: " + str(min(widths)))
Results: Average Height = 388.34, Max Height=914, Min Height = 150
Average Width = 459.12, Max Width = 800, Min Width = 200
IMG_SIZE = 300def load_training_data():  train_data = []  for img in os.listdir(DIR)    label = label_img(img)    path = os.path.join(DIR, img)    img = Image.open(path)    img = img.convert('L')    img = img.resize((IMG_SIZE, IMG_SIZE), Image.ANTIALIAS)    train_data.append([np.array(img), label])    # Basic Data Augmentation - Horizontal Flipping    flip_img = Image.open(path)    flip_img = flip_img.convert('L')    flip_img = flip_img.resize((IMG_SIZE, IMG_SIZE), Image.ANTIALIAS)    flip_img = np.array(flip_img)    flip_img = np.fliplr(flip_img)    train_data.append([flip_img, label])  shuffle(train_data)  return train_data
Test the Image loading to make sure it worked properly
train_data = load_training_data()plt.imshow(train_data[43][0], cmap = 'gist_gray')
model = Sequential()model.add(Conv2D(32, kernel_size = (3, 3), activation='relu', input_shape=(IMG_SIZE, IMG_SIZE, 1)))model.add(MaxPooling2D(pool_size=(2,2)))model.add(BatchNormalization())model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))model.add(MaxPooling2D(pool_size=(2,2)))model.add(BatchNormalization())model.add(Conv2D(64, kernel_size=(3,3), activation='relu'))model.add(MaxPooling2D(pool_size=(2,2)))model.add(BatchNormalization())model.add(Conv2D(96, kernel_size=(3,3), activation='relu'))model.add(MaxPooling2D(pool_size=(2,2)))model.add(BatchNormalization())model.add(Conv2D(32, kernel_size=(3,3), activation='relu'))model.add(MaxPooling2D(pool_size=(2,2)))model.add(BatchNormalization())model.add(Dropout(0.2))model.add(Flatten())model.add(Dense(128, activation='relu'))#model.add(Dropout(0.3))model.add(Dense(2, activation = 'softmax'))
Training Log with this model:
Evaluate model with:
loss, acc = model.evaluate(testImages, testLabels, verbose = 0)print(acc * 100)
Initial Model = 72.09% Accuracy
There are 5 strategies that I think would be the most effective in improving this test accuracy score:
As we see from the training report, this model achieves 100% accuracy on the training set. To go from 100% in training to 72% in testing demonstrates a clear problem with overfitting. Overfitting can be solved by adding dropout layers or simplifying the network architecture, (a la bias-variance tradeoff). However, this is still more of an art than a science and it is very hard to calculate exactly how to reduce overfitting without trial-and-error experimentation.
We could experiment with removing or adding convolutional layers, changing the filter size, or even changing the activation functions.
Convolutional networks work by convolving over images, creating a new representation, and then compressing this representation into a vector that is fed into a classic multilayer feed-forward neural network. We can try adding more hidden layers or altering the number of neurons in each of these hidden layers.
In this tutorial, we simply augment images with horizontal flipping. But you could try other methods such as random cropping, translations, color scale shifts, and many more. If you are feeling ambitious you could also experiment with Neural Style Transfer or Generative Adversarial Networks for data augmentation.
When I looked through this dataset, it was quite obvious that there is a lot of noise in these images that might confuse a Convolutional Neural Network. Therefore, a great strategy to improve this network would be to train an object recognition model to detect pictures of dogs and crop out the rest of the image such that you are only classifying the dog itself, rather than the dog and everything else in the background.
If you are interested in more details on Improving your Image Recognition Models, please check out this article:
towardsdatascience.com
Hopefully, this article helps you load data and get familiar with formatting Kaggle image data, as well as learn more about image classification and convolutional neural networks. This was my first time trying to make a complete programming tutorial, please leave any suggestions or questions you might have in the comments. Additionally, please leave a clap if this article helps you out, thank you for reading!
Connor Shorten is a Computer Science student at Florida Atlantic University. Research interests in deep learning and software engineering.