Introducing DecaVision to train image classifiers with Google’s free TPUs
By: Yan Gobeil
Tags: Decathlon, Image Classification, Artificial Intelligence, Keras, Deep Learning

The only remaining step in the preparation of the data is to transform the images into tfrecords. This a special data format that was created to play well with Tensorflow and it is essential to access the maximal speed improvements that TPUs can provide. Converting the data to tfecords could be done locally as well, but we would have to upload the resulting files manually to Google Storage so we prefer doing it directly on colab.

In a colab notebook, we start by installing and importing the package as we did above. Be careful not to activate the TPU for the data preparation step since Tensorflow is not able to access local files with a TPU. Next we upload our zip file to the notebook’s local storage (we could also use the function decavision.utils.colab_utils.download_dataset to load the dataset from Google Drive) and unzip it using

!unzip image_dataset.zip

As we just mentionned tensorflow cannot access local files when working with a TPU. This is why we have to go through Google Storage to train. It’s at this step that we have to create a bucket to store the tfrecords. The advantage is that the data will be stored there forever and will not have to be loaded every time we restart the colab notebook. To let colab access the GCS bucket, we have to authenticate our Google account:

decavision.utils.colab_utils.authenticate_colab ()

We then simply use DecaVision to create the training and validation tfrecords and upload them directly to our bucket:

generator = decavision.dataset_preparation.generate_tfrecords.TfrecordsGenerator()

generator.convert_image_folder(img_folder='image_dataset/train', output_folder='gs://myproject/image_dataset/train')

generator.convert_image_folder(img_folder='image_dataset/val', output_folder='gs://myproject/mage_dataset/val')

The previous content of the bucket will be deleted to create the new files. By default, the images are split into 16 different files, but we can change the number of shards accordingly to create files of around 100mb. The images are resized to (299, 299) by default, which is the correct size for most of the pretrained models. However we always need to make sure that we use the correct size for the model we want to try.

Training a model

Now that all of the data processing is done, good job! We don’t need to repeat it again. We simply authenticate our Google account when we open a notebook and start training with the data saved directly on Google Storage. For optimal results, we can now activate the TPU.

When training a deep neural network, there are many hyperparameters that need to be tuned in order to obtain the best model, for example the learning rate or the number of epochs. DecaVision does all this testing for us with a single function:

classifier = decavision.model_training.tfrecords_image_classifier.ImageClassifier(tfrecords_folder='gs://myproject/image_dataset', batch_size=256, transfer_model='B5')

classifier.hyperparameter_optimization(num_iterations=25, n_random_starts=10)

This function starts by training a model 10 times with random combinations of hyperparameters. It then uses what it learned from these random combinations to find 15 better ones (hopefully). All these tests are printed with the results so we can choose our favorite combination at the end. Each step of training consists in training an extra layer added on top of a frozen pretrained model and then unfreezing the last block of that model to finetune it. To learn more about this technique, called transfer learning, refer to the article series mentionned in the introduction (in particular part 1)!

The only options to choose when doing hyperparameter optimization with DecaVision are the batch size and the pretrained model. In this case, we are taking EfficientNet B5, but many others are available, like ResNet, Xception and EfficientNet B3.

Once the hyperparameters that generate the model with the best validation accuracy are found, we need to train the final model. The hyperparameters that are dictated by the optimization are the number of epochs, a distinct learning rate for each of the two steps of training, the dropout rate, the number of units in the extra layer and whether of not to do the finetuning step.

classifier = decavision.model_training.tfrecords_image_classifier.ImageClassifier(tfrecords_folder='gs://myproject/image_dataset', batch_size=256, transfer_model='B5')

classifier.fit(hyperparameters)

Here we do not show the hyperparameters that were found for the yoga poses classifier. The final step in the training portion is to save the model. This is as easy as using the following method:

classifier.model.save('model.h5')

If the extension .h5 is not specified, the model will be exported to a SavedModel format. This format can however only be used on a TPU if we save the model to GCS.

TPU vs GPU vs CPU

Before moving along, we would like to report the results of a small experiment that we did to make clear the fact that using colab’s TPUs is worth all the trouble. We trained a model with the same hyperparameters (except batch size, which has to be higher for TPUs and lower for GPUs for performance) on our yoga dataset (around 11k images) and tried the different processing units available on colab pro. Here are the results:

The TPU available was the v2 and we averaged around 40 seconds per epoch on both steps of the training. The GPU that we tried was the Tesla v100 and we averaged only 55 seconds per epoch for the first step of training and 70 seconds for the fine tuning step. Finally, just for fun, the first epoch took more than 30 minutes on the available CPU…

Evaluating the model

Once a satisfactory model has been saved, it is important to use it a little before declaring victory. We can use DecaVision to evaluate the model with different tools. The simplest test to do is evaluating the accuracy on different datasets where the images are separated in a folder per category:

tester = decavision.model_testing.testing.ModelTester ('model.h5')

tester.evaluate(path='image_dataset/val')

The results that we got were an accuracy of 96% on the training set, 95% on the validation set and 83% on the test set. The model performs much worse on the test set because it was made only from Instagram images, which are harder than the Google images that make up a part of the other two datasets.

To see explicitely how our yoga classifier performs on new images that it has not seen we can just place them in a folder and use the following function:

import os

categories = os.listdir('image_dataset/train') tester. classify_images (image_path='image_dataset/test', categories=categories)

This feature requires an ordered list of the possible categories, which we infer here from the training dataset itself. Here are a few examples of results for the yoga poses classification model: