Tile (Slice) YOLO Dataset for Small Objects Detection | by Rostyslav Neskorozhenyi  | Towards Data Science

In one of my projects I was faced with the task of detecting very small cracks in very high resolution images. I was provided with an already labeled dataset (in YOLO format), and the simplest solution would be to train the object detection model on it as is. But such an approach would require a lot of computing power and a lot of time. The alternative option to reduce the images (say to a size of 416x416) resulted in a serious loss of detail and accuracy (some cracks just disappeared).
Finally, I came up with a simple solution that allowed me to save computing resources and preserve accuracy. I sliced each image (and its corresponding labels) into small tiles (the same 416x416) and trained the model on the resulting tiles. The resulting detection on a high-dimensional image was collected from several detections on tiles from which this image was composed.
To cut the dataset, I wrote a small script, which I want to share with you. In this article I will briefly describe the algorithm of the script. You can get the script itself from the link on github. So let’s get started.
We will need pandas to load and save the dataset, numpy for working with arrays, PIL for working with images, shapely for labels slicing, glob for listing images in a folder, and matplotlib for visualization.
import pandas as pdimport numpy as npfrom PIL import Imagefrom shapely.geometry import Polygon, Pointfrom matplotlib import pyplot as pltimport glob
We assume that we have a dataset in YOLO format, which consists of a folder with images and text files with labels corresponding to these images. For example, if I have an image named im1.JPG, the corresponding labels will be stored in the im1.txt file. The file with labels follows this syntax, for each bounding box the file will contain a line in the format: <object-class> <x_center> <y_center> <width> <height>. We also need a classes.names file for human-readable class names, as well as train.txt and test.txt files to split the dataset into train/test parts. In my case, I have only one class - "crack". We will implement the train/test split further.
The structure of our dataset at this stage should look like this:
yolosample/    classes.names    ts/        im2.txt        im1.txt        im1.JPG        im2.JPG
I recommend reading more about the dataset format, as well as YOLOv4 training on your custom dataset in this excellent article.
Let’s visualize labels on one of the images. First, let’s open the image and the corresponding labels.
imname = 'im1'im = Image.open(f'yolosample/ts/{imname}.JPG')df = pd.read_csv(f'yolosample/ts/{imname}.txt', sep=' ', names=['class', 'x1', 'y1', 'w', 'h'])imr = np.array(im, dtype=np.uint8)
Rescale the coordinates from labels according to the dimensions of the image, since they are now normalized from 0 to 1.
df_scaled = df.iloc[:, 1:]df_scaled[['x1', 'w']] = df_scaled[['x1', 'w']] * imr.shape[1]df_scaled[['y1', 'h']] = df_scaled[['y1', 'h']] * imr.shape[0]
We can now display labels on the image.
import matplotlib.patches as patches# Create figure and axesfig,ax = plt.subplots(1, figsize=(10,10))# Display the imageax.imshow(imr)for box in df_scaled.values:    # Create a Rectangle patch    rect = patches.Rectangle((box[0]-(box[2]/2),box[1]-(box[3]/2)),box[2],box[3],linewidth=2,edgecolor='g',facecolor='none')# Add the patch to the axes    ax.add_patch(rect)    plt.show()
As we can see, the objects are very small and difficult to distinguish. This is where the slicing comes in handy.
In this script, we slice the image into tiles of slice_sizeXslice_size and save these tiles in the newpath folder. We use shapely to find the bounding boxes that intersect with each specific tile and store the intersection in the labels file for the corresponding tile. We will also save the tiles with no bounding boxes in the falsepath folder.
Now we can see what we got.
imname = 'im1_2_10'im = Image.open(f'sampletiled/ts/{imname}.JPG')imr = np.array(im, dtype=np.uint8)df = pd.read_csv(f'sampletiled/ts/{imname}.txt', sep=' ', names=['class', 'x1', 'y1', 'w', 'h'])# rescale coordinates for visualizationdf_scaled = df.iloc[:, 1:]df_scaled[['x1', 'w']] = df_scaled[['x1', 'w']] * imr.shape[1]df_scaled[['y1', 'h']] = df_scaled[['y1', 'h']] * imr.shape[0]# create figure and axesfig,ax = plt.subplots(1, figsize=(10,10))# display the imageax.imshow(imr)for box in df_scaled.values:    # create a Rectangle patch    rect = patches.Rectangle((box[0]-(box[2]/2),box[1]-(box[3]/2)),box[2],box[3],linewidth=3,edgecolor='g',facecolor='none')# add the patch to the axes    ax.add_patch(rect)    plt.show()
Everything looks good and now we just need to split the dataset into train/test parts.
import shutilimport osimport random# get image names and set train/test split ratioratio = 0.8imnames = glob.glob('sampletiled/ts/*.JPG')names = [name.split('/')[2] for name in imnames]# split dataset for train and testtrain = []test = []for name in names:    if random.random() > ratio:        test.append(f'sampletiled/ts/{name}')    else:        train.append(f'sampletiled/ts/{name}')print('train:', len(train))print('test:', len(test))
Save train.txt, test.txt and classes.names for a new dataset. In my case I have only one class. You can specify any needed classes.
# save train partwith open('sampletiled/train.txt', 'w') as f:    for item in train:        f.write("%s\n" % item)# save test partwith open('sampletiled/test.txt', 'w') as f:    for item in test:        f.write("%s\n" % item)# save classes namesclasses = ['crack']with open('sampletiled/classes.names', 'w') as f:    for item in classes:        f.write("%s\n" % item)
Congratulations! We have implemented a tiling script and with its help generated a new dataset on which our model can learn to better detect small objects. As I already mentioned, you can find the final script on github by the link. Yo can also find this article packed in Google Colab by this link. If you liked the article and the script was useful for you, then do not forget to clap for this post, I will appreciate it.
Good luck with Computer Vision!