Google Cloud AI Platform: Human Data labeling-as-a-Service Part 1
By: James Green
Tags: Image Recognition, Image Classification, Google Cloud Platform, Data Science, Machine Learning

Data Labelling Service on Google Cloud

Google’s Data Labelling Service provides access to human labellers to generate highly accurate labels for training data.

Currently in public-beta, the service formed part of the original AI Platform launch in April 2019.

The labelling service currently supports 3 media types:

Images Video (we will cover in part 2 of this series) Free-text (we will cover in part 3 of this series)

We will now explore image labelling, including an easy to follow, technical walkthrough.

1. Image labelling

Image labelling is the most common use-case we come across in our client projects. The service covers a good range of image labelling tasks:

Classification

“CONTAINS CAT” (Photo by Max Baskakov on Unsplash)

“NO CAT” (Photo by Karsten Winegeart on Unsplash)

Image classification is the simplest to do, and a good place to start if you are using the service for the first time. I’ll use the above two images as an example of how easy it is.

First, upload your images to be labelled to a Google Cloud Storage bucket. Here, you can see the above images uploaded to a bucket called cats-or-no-cats.

Author Image

Tip: This applies to any image labelling. Make sure your training images are as close as possible to the images the model will be predicting on. For example, think about resolution, colour/black and white, avoid commonalities in backgrounds (as in our wolf example earlier) that can confuse the model.

2. Next you need to create a .CSV file that contains the URIs to your images in your GCS bucket. So, in my example, my CSV just contains two lines:

gs://cats-or-no-cats/cat.jpg

gs://cats-or-no-cats/no_cat.jpg

Upload this .CSV file to the same bucket that contains your image files.

3. Next we need to create data labelling dataset. This dataset is simply a resource that the human labellers will refer to for your lablleing request. It contains your images, image .CSV file and your instructions (I’ll cover instructions shortly).

To create a dataset, first, navigate to the Data Labelling Service UI in the Google Cloud console. You should see a screen like this (enable the API if you are prompted).

Google Data Labelling Service in the Google Cloud Console (author image)

Click the CREATE button. Then, enter the details of your new dataset in the panel, shown below. Note I have to choose the CSV file in my cats-or-no-cats GCS bucket. Finally, click CREATE.

So simple! (image author)

I now see my dataset with both of my images imported. Google documentation states this step takes approximately 10 minutes per 10,000 images uploaded.

My new dataset, ready for labelling (author image)

4. Next we create a label set, to define the label options we want the human labels to label our images with. In our example, we need two labels: “Cat” and “No Cat”.

To do this, from the Data Labelling UI click Label Sets from the menu on the left and then click CREATE at the top of the screen. Then, in the panel on the right, simply define your labels. Here are mine:

Defining your label sets (author image)

Tip #1. We always include a clear, unambiguous description with each label. Labellers will see these, and therefore these supplement your instructions! Tip #2. For multi-classifcation use-cases (such as cats or dogs) we always include a label for “Both” (image has a cat and dog) and “None” (image has no cats or dogs). As well as, of course, “Cat” and “Dog” :)

5. The penultimate step is to create an instruction document. This is a PDF document that helps human labellers to understand what you want them to label. Now, for our classification task this is really simple and just requires a few example images (positive and negative) under each of our labels.

Google has a lot of detailed guidance on writing good instruction documents. We would recommend reading this, as if these instructions aren’t clear, your labellers may not do what you want them to do!

Once your PDF document is ready, upload it to your GCS bucket that contains your images and .CSV file. Next, click Instructions in the UI and in the panel on the right just navigate to your instructions document. Here’s what mine looks like:

Uploading instructions for human labellers to help understand your task (author image)

Click CREATE INSTRUCTIONS and wait for an email confirmation before continuing. Here’s my email (it usually only takes a few seconds).

My instructions are ready! (author image)

6. We are now ready to submit our labelling task! This is really straightforward — navigate to your dataset (see step 3) and click on the dataset name. You will see your dataset contents. Here are mine:

My new dataset, ready for labelling (author image)

Click on CREATE LABELLING TASK at the top. In the panel that’s displayed enter:

Enter a name for the annotated dataset (this is the dataset that will contain your human labelled images). Add a description too.

From the objective dropdown, I choose Classification .

dropdown, I choose . From the label sets dropdown, I choose my Cats-or-no-cats label set.

dropdown, I choose my Cats-or-no-cats label set. From the instructions dropdown, I choose my instructions PDF from my GCS bucket.

dropdown, I choose my instructions PDF from my GCS bucket. Next, I choose how many people I want to review the labelling of each image. The default is one, which we typically choose. However, if your tasks demand extra rigour you can increase this to three or five.

Lastly, accept the terms and conditions and click Create.

Congratulations! Your images will now be labelled by skilled human labellers. Pretty cool, hey?

Reviewing your labelled images

Once the labelling process is complete, you will receive an email confirmation. To review your labelled images, first, navigate to your dataset. Here is mine:

My dataset, labelled and ready for review (author image)

Click on the LABELLED DATA SETS link and you will see your labelled images. Exporting labelled images is a really useful feature, especially if your image training set is large. Options are to exports as a CSV or json file. To export your images:

From the LABELLED DATA SETS page click EXPORT In the dialogue that appears, select a target GCS folder and the format (we usually choose CSV).

Tip: Both CSV and json exports can easily be viewed in BigQuery (via a federated external table) for further analysis.

And that concludes my walkthrough.

As well as classification, Google’s Data Labelling Service covers the following other image labelling tasks:

Bounding box labelling

In bounding box, we want to label a particular feature of an image using a rectangular box. For example, as an extension of our Cats-or-no-cats example, we may choose to train a second model that can detect the position of a cats face within an image.

Bounding Box to detect the location of a cats face in an image (Photo by Max Baskakov on Unsplash)

Tip. Pay close attention to your instructions document and make sure it’s really clear to labellers what you want (and don’t want) boxed in your images.

Oriented bounding box labelling

This is similar to the bounding box task, with the only difference being the boxes can be orientated (angled). We haven't come across this in the field, but this paper has some good examples relating to the identification of cars in aerial photography.

Bounded polygon labelling

This is useful if more precision is needed in object detection than bounded box. In this task, human labellers will label an image with closed polygons (comprising of a series of co-ordinates).

Let’s say we want our cat to be bounded by a polygon. Here’s what the labelled image might look like:

Bounded polygon image labelling (Photo by Max Baskakov on Unsplash)

Polyline labelling

Polyline is similar to Bounded Polygon, with the exception the polygon does not need to be closed. This is often used for labelling roads in images, but if we continue our cat example, we could use a polyline to mark the cat’s mouth (perhaps we want to analyze if a cat is happy!).

Polyline labelling of a cat’s mouth (Photo by Max Baskakov on Unsplash)

Image Segmentation labelling

This form of labelling splits an image into a series of segments, each labelled with a colour. It’s frequently used in driverless car AI, where a street scene must be separated into different segments (road, pavement, cars, buildings, sky, etc).

In our example, we simply use this to label segments comprising cats.