Create A Synthetic Image Dataset — The “What”, The “Why” and The “How” | by Viraf | Towards Data Science

It is quite a known fact that the quality and quantity of dataset used to train your model will have a direct effect on the model’s performance. Availability of an extensive, versatile dataset could seal the deal, and you could jump to the next step of your Machine Learning / Deep Learning pipeline. But often you might end up in a situation where the dataset you need is just not readily available. Or the data you have is not enough to train a heavy duty model. Now what do you do?
There are so many resources out there that can help you create a new custom dataset of any object class you need. But if I had to describe the process of building a dataset in one word, I’d call it “cumbersome” and “time-consuming” and “frustrating” and — I know I used more than one word, but you get the point!
Especially when it comes to drawing segmentation masks, I just can’t be bothered to annotate any more than a few images! Don’t get me wrong, I know dataset creation is a very very very important step, but isn’t there a way out?
Data augmentation is a great way to increase the variety of the dataset that your model sees, but again, there is a limit to how much it can help. How about we take it a step further?
Creating any kind of data (image, audio, text, etc.) ‘artificially’ or ‘programatically’ results in what we call a synthetic dataset. This data is not collected naturally or normally and labelled manually, so is it safe to use it to train your model? Will it give good results?
While there are many papers claiming that carefully created synthetic data can give a performance at par with natural data, I recommend having a healthy mixture of the two. A well designed synthetic dataset can take the concept of data augmentations to the next level, and gives the model an even larger variety of training data. The more the better right?
There are several ways out there to create synthetic image datasets, and in this tutorial, let’s look at one of the most basic method — Image Composition. We will generate new images by composing the object image as a foreground on multiple background images.
In this tutorial, I am going to create synthetic images for the output class “dog”. In reality, there are many existing datasets online that include the dog class (for example, COCO), but what’s the harm in creating more pictures of these cute gifts of nature. :P
Have a look at the import statements below. Install any module you might not already have.
import osimport numpy as npimport skimage.io as ioimport skimage.transform as transformimport matplotlib.pyplot as pltfrom matplotlib.patches import Rectangle%matplotlib inline
There are 2 ways to go about this —
(A) Create your own dataset
First, let’s take an image of a dog available on the internet. There are several websites out there which allow you to use the images for free like — pexels, flickr, unsplash — take your pick. I’m going to use the one below.
Next, you will have to put in a little manual effort to separate the dog from the background. Yeah, I’m sorry — there are no free meals. But eventually this is definitely lesser effort than creating an entire dataset, I promise.
A few tools that I know of to get this job done are Photoshop (paid) or GIMP (free). Or you could even use the ever-dependable Microsoft Paint! The final output looks like the one below.
Read the image. Next, cut out the foreground layer from this image and set the remaining pixels to zero. An alternative to coding this step is to add a black background with the tool you used (GIMP/Paint/Photoshop) while segmenting the image itself. But nonetheless, here’s how to extract the foreground object in python — simple thresholding of pixels.
# Read the imageI = io.imread('./dogSeg.jpg')/255.0# Cut out the foreground layerforeground = I.copy()foreground[foreground>=0.9]=0 # Setting surrounding pixels to zeroplt.axis('off')plt.imshow(foreground)plt.show()
(B) Use an existing dataset
If you don’t want to start from scratch, another option is to make use of the images from an already available dataset.
I am going to use the COCO dataset here to demonstrate. Since COCO already has the object category “dog”, it will be easy to retrieve the image and its mask. The COCO dataset has 4385 (train) and 177 (val) images for the “dog” category.
You can refer to the entire code for this process in my GitHub repository for this tutorial. I will not be going over the code for this step in this post.
If you are interested in manipulating the COCO image dataset and creating a data generator to train your model for Image Segmentation, I have an extensive tutorial here.
Let’s play a bit with the dog (pun intended). I have used random rotations, random zoom, random translations and random horizontal flips in my code, but you can use something else too.
The output of the code is shown below. See how different it looks from the original image!
After applying all these changes to the foreground, let’s extract the segmentation mask from the image.
Now you need a new background image. Use the free websites I mentioned earlier to get a wide variety of backgrounds. You can choose any image in the world (see how I sent my dog to the moon in the display picture of this article), but I recommend keeping it as realistic as possible.
I am going to use these 4 images as my backgrounds for this tutorial.
Let’s select a background at random.
# Random selection of background from the backgrounds folderbackground_fileName = np.random.choice(os.listdir("./backgrounds/"))background = io.imread('./backgrounds/'+background_fileName)/255.0
It’s time to add your dog image to the background!
This function gives the following output. Now, I know it’s not perfect, but it’s quite good!
For object localization tasks, you can also get the box coordinates easily from the new segmentation mask we developed.
And on performing these functions in a loop with all the backgrounds, we get a nice set of images!
In this tutorial, I used 1 foreground with 4 backgrounds. If you use multiple foregrounds in random with multiple backgrounds, you can create many, many images! Maybe you could merge two or more foregrounds on a background as well. Get creative — play with sizes, angles, foregrounds, and backgrounds to build an awesome synthetic dataset according to your needs. There are a lot of design considerations here as well, but with a little effort you can do much more and create more accurate datasets.
Finally, you can find the entire code for this tutorial in my GitHub repository.
As a next step on this topic, check out this nice tutorial on creating synthetic image datasets with Unity 3D.
Well, that’s it for now — thank you for reading this far into the article. I hope something good came out of it. Any thoughts, questions, comments? Let me know in the responses below!
If you are interested in image datasets for semantic segmentation, check out my post below:
towardsdatascience.com
Or do you want to be rich overnight using ML in stocks? This article is (NOT) for you!
towardsdatascience.com
Can an ML model literally read the stock price charts?