## 3 Materials and Methods
### Dataset Generation  
Most DL models are known as supervised ML algorithms. This specificity implies that the training mechanisms for DL models need to be fed with an annotated dataset. In the case of object detection, each annotation identifies its class, size and position. Some  
Figure 5: Greenhouses’ entrance. Figure 6: AgRob v16 inside an uncultivated greenhouse. annotation types, like the Pascal VOC format, also include some additional features of the annotations as whether the target object is difficult to detect, occluded or truncated. For the current dataset, we used the Pascal VOC format, as used in the Pascal VOC Challenge [52], due to its simplicity, resuming the annotations for each image in a single XML file. In this case, we ignored the additional features because the TensorFlow1 Object Detection Pipeline ignores these features. First of all, we converted the continuous video with the images of tomatoes into individual sequential images. To avoid high repetitiveness between images in the dataset, we acquired a frame every three seconds, assuring an overlapping ratio of around 60%. This process resulted in a dataset of 297 images with a resolution of 1280 \(\times\) 720 px each. All the images were manually labelled using the annotation tools CVAT [53] and LabelImg [54]. These tools allow better management of images and annotations, as well as a collaborative annotation. For this dataset, we only considered the class \"tomato\", ignoring their ripeness, because of the low amount of ripened tomatoes (Figure 1), i.e., most of the tomatoes in the dataset were reddish or green (Figure 1). We intended to use TPUs to detect tomatoes online inside the tomatoes' greenhouses. However, TPUs are not compatible with all the ANN models. At the current time, they are only compatible with SSD models [11] and the tiny versions of You Only Look Once (YOLO) [42] model. Additionally, these models cannot process full-sized images, rescaling all the images before processing them. For instance, the pre-trained MobileNet v2 [15] can only process images of 300 \(\times\) 300 px. Due to this, we split the original images into 300 \(\times\) 300 px images, following the scheme represented in Figure 7, using pascal_voc_tools (see [[https://github.com/wang-tf/pascal_voc_tools](https://github.com/wang-tf/pascal_voc_tools)]([https://github.com/wang-tf/pascal_voc_tools](https://github.com/wang-tf/pascal_voc_tools)), accessed on 8 September 2021). This splitting scheme considered the construction of sub-images of 300 \(\times\) 300 px with a minimum overlapping ratio of 20% between sequential images. Splitting full-sized images of 1280 \(\times\) 720 px to 300 \(\times\) 300 px increased the dataset and the quality of the considered images for training. Therefore, this procedure increased the accuracy of the developed system. The Dataset now had 5365 images. Some researches concluded that using augmentation strategies in the original images allowed increasing the dataset size and variability, contributing new information to it [55]. Different kinds of transformations may be applied to an image:  
1. rotation;
2. translation;  
Figure 7: Images split into 300 \(\times\) 300 px images with an overlapping ratio of 20%. The different colours are only for reference and distinguishing the different images. * (iii) scaling;
* (iv) hue modification;
* (v) saturation;
* (vi) blur;
* (vii) noise;
* (viii) others, even combinations of transformations. Table 2 resumes the different applied transformations to images and their specifications. All transformations were applied with a random factor, to increase the variability of the data. Figure 8 exemplifies the use of augmentation in images. The augmented dataset resulted in a total of 23,021 images with 61,204 annotations of tomatoes. Figure 8: Example of augmentation applied to an image. (**h**) is the random combination of 3 of the other transformations. For training purposes, we divided the dataset into two sets: training set and validation set.