How to Leverage GCP’s Free Tier to Train a Custom Object Detection Model With YOLOv5
By: Julian Lopez Baasch
Tags: Computer Vision, Object Detection, Yolov5, Google Cloud Platform

How to Leverage GCP’s Free Tier to Train a Custom Object Detection Model With YOLOv5

In this tutorial, I’ll show you how to leverage your GPC free tier and get annotated data for training your own model. We’ll see how to build a strong baseline despite using a small dataset (~800 samples!).

Let’s first take a look at how our resulting model infers on unseen data (remember, only 800 samples were used):

GCP offers $300 credit in a 3-month free trial so you can explore and conduct an assessment of its platform. Basically, you can use any of their resources, like BigQuery, GPU instances, Cloud Storage, and (in particular) Cloud AI. This last product offers pre-trained API models for different tasks, like object detection, image classification, emotion recognition, language translation, and so on. In this post, I’ll show you an example with the Vision API, and in particular, its logo detection feature.

Throughout the tutorial, I’ll provide with useful gists to facilitate each step. However, I won’t get much into detail, so I recommend you to get familiar with both GCP and YOLO.

Basically, YOLO (You Only Look Once) is a fast and light model for object detection. Although is not the most accurate model in this task, it gained considerable popularity among practitioners due to its lightness. The first version came in 2016, with a 24 convolutional layer followed by 2 fully connected layers architecture, and from then it has been improved through 4 different versions. This year, YOLOv5 got open-sourced by Ultralitycs, and the complete project is shared on their Github.

YOLOv1 Architecture (source: https://medium.com/r?url=https%3A%2F%2Farxiv.org%2Fpdf%2F1506.02640.pdf)

1) Get your data

Once you created a GCP account you need to enable Cloud Vision and Cloud Storage APIs to build a pipeline to download and process your data before training a model. Also, you need to authorize the APIs within your GCP Project and download the credentials in JSON format.

The next step is to download an interesting video from YouTube to get data from. In our example, I used Formula 1’s videos, in which sponsors and logos are almost protagonists on screen.

After downloading the video file into our local disk (it can be either in .mp4 or .mkv formats), we upload it to our desired bucket in Cloud Storage. This step is optional, as you can also use VideoIntelligence API with locally hosted videos, although this makes the API call way slower.

Now, we can call the VideoIntelligenceAPI to process video annotations for logo recognition. It takes it’s time, even with a 10min long video. But don’t worry, VideoIntelligence prices are quite low compared to other Google AI services. So grab a coffee and wait for your video to process :).

The API responses in a JSON alike annotation object, which you need first to serialize it in case you want to store it for later use. Since the annotations are hard to interpret, we build the following parser so that we can convert this information as a pandas DatafFame:

Below I display an example of the parsed data into a DataFrame. VideoIntelligence API provides detailed annotations of bounding boxes in seconds and microseconds, together with the confidence of its prediction. And what the anno_to_df.py does is to populate this DataFrame with the corresponding video frame at every instant of the video.

Besides the video, we need to extract frames from it and store them on our local disk. The idea is that we can use these frames images as inputs for our object detection model, while the output would the annotation given by Google's Vision AI. We’ll be building the dataset in the following steps.

2) Process Vision AI annotated data

But to train a model, not only do we need the bounding boxes of logos, but we should also provide the image path to each video frame. Additionally, we have to deal with the fact that our model needs also examples where logos weren’t detected. That is, if no logo is detected between frame 10 and frame 15, we should fill the DataFrame with 5 samples (rows) with bounding boxes set to 0 and class equal to None.

Then, we arrive at a DataFrame similar to this one: