How to algorithmically generate an image dataset | by Albert Sanchez Lafuente | Towards Data Science

This post is inspired by the NFT art collections that have been released lately, such as Cryptopunks, Sollamas, ON1 Force, or Bored Ape Yacht Club. Among all these projects, there are some in which the images are created by artists while in others the images are algorithmically generated. As we are in a Data Science blog, the fun way for us to create a collection of images is the second one.
In this article we are going to create, using Python, a collection of random images with many features in common between them. We will start from some base images to which we will be adding elements on top to configure the final images that will form the collection. In this case we are going to use a background on which we will place different types of characters on which we will add an object.
Finally we will have a dataset of images in which each image will have specific attributes. For example a pirate in a desert with a donut, or a monster in the forest with a key.
As the goal is not to create art, I will take freely distributed pixel art as a base. I will use these images as a base to generate hundreds of derivative images. As a fun exercise you can pick some different images and create your own totally different dataset.
The attributes I will use will be three types: the background, the character and the objects. For each of them I will set a probability distribution in the form of a percentage to define how common each of the features will be. Feel free to play with the parameters to create different kinds of datasets. For example a dataset in which 99% of images use one feature and 1% another. By doing experiments of this kind and creating a predictor model on top of them you can learn a lot.
To generate the dataset of the article I am going to use this property distribution:
The first step is to find the images that will be used as a base. Using a wide variety of websites, I have selected some images to use as a base. The main criterion for the selection of the images is that they can be modified and distributed free of charge according to their license. You can download all the images used in this repository.
Once the images have been selected we can start programming the algorithmic image generator. To generate the images I will use Python together with the Pillow library. This is the basic script to generate an image:
from os import path, mkdirfrom PIL import Imageoutput_folder = "generated"if not path.exists(output_folder):mkdir(output_folder)background_file = path.join("backgrounds", "forest.png")background_image = Image.open(background_file)output_file = path.join(output_folder, "generated1.png")background_image.save(output_file)
First the “generated” folder is generated to save the generated files. If the folder already exists, we do nothing. We load one of the backgrounds and save it in the “generated” folder. We have done our first operation with images! Now it’s just a matter of adding more elements on top until the final images are generated.
We will give fixed dimensions to the backgrounds so that all generated images have the same dimensions. To simplify the work, I have scaled the largest background images to the same size as the smallest of them. I have not scaled the characters because the goal of this project is to be functional, not pretty. Some objects I have scaled because they were significantly larger than the others and they were out of the image.
The background images used have dimensions of 1920x1078 px once scaled.
On characters and objects the only transformation made has been to eliminate the margins since each image had different margins on each side. This made that even when positioning them in the same position some of them were higher or more to the right than others.
The work gets a little more complicated when adding the characters and objects on top of the background. Now we have to define in which position we want to place the characters and how big they should be. To define the position we have several options:
- define a fixed position for all images- define a fixed position for each type of character / each type of landscape- randomize the position. If we choose the latter we should define some limits where we don’t want the images to appear, we don’t want an image to be placed on a border and only a small part of it to be seen.
With objects it will be the same, we will have to define where to place them depending on where the character is or in a predefined position on the background. For this dataset I have decided to place the objects to the right of the characters.
Pillow uses a coordinate system that has origin in the upper left corner of the image. As seen in this image:
To simplify the work I have decided to place all the characters in the same fixed position on the background. I have defined the position as a point centered horizontally at the bottom of the image. The dimensions of the small rectangle, h and w are the height and width of the character. The point marked in black is the one we have to calculate to tell Pillow where to place the character in the image.
This point can be calculated with the following formula, where 1920 is the width of the background that we divide by 2 to find the midpoint. 1000 is an approximation to 1078 to leave some margin between the character’s feet and the bottom of the image.
To place the objects to the right of the characters, I use a similar formula, in this case I add a margin m so that the object is not touching the character. A value of m that worked well for me was 30 px. Attention, h and w in this case refer to the height and width of the character, not of the object since we are positioning to the right of the character.
All these calculations translated into code become this:
def generate_image(background, character, object, file_name):  background_file = path.join("backgrounds", f"{background}.png")  background_image = Image.open(background_file)  #Create character  character_file = path.join("characters", f"{character}.png")  character_image = Image.open(character_file)  coordinates = (int(1920/2-character_image.width/2), int(1000-character_image.height)) #x, y  background_image.paste(character_image, coordinates, mask=character_image)  #Create object  if object != "none":  object_file = path.join("objects", f"{object}.png")  object_image = Image.open(object_file)  coordinates = (int(1920/2+character_image.width/2+30), int(1000-object_image.height)) #x, y  background_image.paste(object_image, coordinates, mask=object_image)  output_file = path.join(output_folder, f"{file_name}.png")  background_image.save(output_file)
At the same time that we generate the images, we’ll also register them in a database together with their characteristics. We’ll use pandas to do the registration and we’ll save the data in a CSV file.
We create a DataFrame without data, only naming the columns. Each time we generate an image, a new row will be added to the DataFrame. Finally, once all the images have been generated, the CSV will be saved. The num variable is a sequential counter used to name the images. For example, the generated images will be named generated1, generated2, generated2894, ....
It’s important to save the name of the image file to know to which file these parameters correspond.
df = pd.DataFrame(columns = ["background", "character", "object", "generated image"])for ... .... ....  #Code to generate image  data = [background, character, object, f"generated{num}"]  s = pd.Series(data, index=df.columns)  df = df.append(s, ignore_index=True)df.to_csv('data.csv', index=False)
You can calculate the total number of possible images with a simple formula. Just multiply the number of possible attributes in each category by each other. In our case, 4 backgrounds * 5 characters * 11 objects = 220 images.
To generate all possible images we will use this code. First we define all possible values for each of the parameters, then we iterate over them.
backgrounds = ["countryside", "desert", "forest", "glacial"]characters = ["mage", "warrior", "pirate", "monster", "toadking"]objects = ["none", "barrel", "anchor", "axe", "bomb", "key", "chest", "bananas", "cupcake", "donut", "heart",]def generate_all_imgs():  num = 0  df = pd.DataFrame(columns = ["background", "character", "object", "generated image"])  for background in backgrounds:    for character in characters:      for object in objects:        generate_image(background, character, object, f"generated{num}")        data = [background, character, object, f"generated{num}"]        s = pd.Series(data, index=df.columns)        df = df.append(s, ignore_index=True)        num += 1  df.to_csv('data.csv', index=False)
When generating the images, we have seen how to generate all possible images. In this case we can generate them all because they are few, but if we increase the number of parameters or the number of possibilities of each parameter, it’s possible that the number of images to generate is enormous.
With this code we are going to generate a given number (total_imgs) of images. To decide which features to use in each image, we are going to resort to chance. Note that it is possible that the same image is generated several times, and this will be common in the case of the most likely parameters (most likely background with most likely character and most likely object, in our case it could be a mage in a forest without object).
We will use the probabilities defined on the table above for each attribute, and using Numpy’s random.choice function we will choose a property taking into account which one is the most likely.
def generate_random_imgs(total_imgs):  df = pd.DataFrame(columns = ["background", "character", "object", "generated image"])  for num in range(total_imgs):    background = np.random.choice(np.arange(0,len(backgrounds)), p=[0.3, 0.3, 0.3, 0.1])    background = backgrounds[background]    character = np.random.choice(np.arange(0,len(characters)), p=[0.4, 0.3, 0.2, 0.095, 0.005])    character = characters[character]    object = np.random.choice(np.arange(0,len(objects)), p=[0.3, 0.2, 0.1, 0.1, 0.1, 0.05, 0.05, 0.04, 0.03, 0.025, 0.005])    object = objects[object]    generate_image(background, character, object, f"generated{num}")    data = [background, character, object, f"generated{num}"]    s = pd.Series(data, index=df.columns)    df = df.append(s, ignore_index=True)  df.to_csv('data.csv', index=False)
Be careful not to use too high a number because the generation process could take hours and the images created could occupy many GB.
We have created a small and basic program to generate datasets of composite images from basic images. With this dataset we can start to train a neural network to classify the images according to the type of background or according to the type of character.
Also with this script we have the basis to do more complicated things, we could add rotations to the characters, make them have different sizes, change colors, etc...
You can use this dataset generator as a testing playground for data augmentation techniques, for example you could create new images where the character is in another position instead of always the same. You could create images where the character or the object is rotated, etc...
You can find all the code as well as the original images used in this Github repository.
The images used to create the dataset are not mine, their original sources are:
characters and objects: https://craftpix.net/freebies/
glacial background: https://www.publicdomainpictures.net/es/view-image.php?image=282737&picture=lago-glaciar-jokulsarlon
desert background: https://www.publicdomainpictures.net/es/view-image.php?image=342429&picture=autopista-de-arizona
countryside background: https://www.publicdomainpictures.net/es/view-image.php?image=200702&picture=casa-de-campo
forest background: https://www.publicdomainpictures.net/es/view-image.php?image=167694&picture=bosque-de-la-pintura-al-oleo