# Deep Learning-Based Occlusion Handling of Overlapped Plants for Robotic Grasping
## 3 Materials and Methods  
### Dataset  
The dataset of overlapping plant cuttings placed on a flat surface such as a conveyor belt was created using Vaccinium cuttings. Two unique appearances of each cutting, front and back, were used to create different overlap cases of the cuttings on the flat background. The RGB images of the overlapped cuttings with a resolution of 2048 \(\times\) 1536 pixels were captured with a stereo camera set. In total, 650 images were captured, and they were annotated for instance segmentation. Since plants do not have a fixed structure, their overlap can be so complicated that in some cases it is not possible even for human annotators to distinguish between different plant instances. In the context of the present work, several terms are defined to narrow down the cases of occlusion:  
* An occlusion patch is a group of pixels in the image for which one part of a plant cutting is occluded by other plant cuttings and, thus, this part is not visible in the image. Based on this definition, one cutting can have several occlusion patches with a single cutting or multiple other cuttings. * A normal occlusion patch is the occlusion patch in the image for which a part of only one plant cutting is not visible. It should be noted that one plant cutting can have multiple normal occlusion patches. The red windows in Figure 2 illustrate some locations where normal occlusion patches are present. * A complex occlusion patch is the occlusion patch in the image for which parts of more than one plant cuttings are not visible. One plant cutting might have multiple complex occlusion patches or even a mixture of several normal and complex occlusion patches. The purple windows in Figure 2 illustrate some locations where complex occlusion patches are present. * A normal occlusion image is an image that contains at least one normal occlusion patch and no complex occlusion patches. * A complex occlusion image is an image that contains at least one complex occlusion patch. Figure 3 shows examples of normal and complex occlusion images with annotations of plant cuttings as belonging to different object classes. As it can be seen, the following four object classes are introduced: Target Cutting, Occluded Cutting, Singularized Cutting, and Remains. The cuttings that occlude other cuttings and that are not occluded themselves as they are on the top of other cuttings are classified as Target Cuttings. The cuttings which are occluded by other cuttings are classified as Occluded Cuttings. The cuttings that are not occluded and also do not occlude other cuttings are classified as Singularized Cuttings. Individual plant parts such as a single leaf or a stem part or cuttings that have two or fewer  
leaves are classified as Remains. In total, 25% of all images in the presented dataset are complex occlusion images. The overlap of sum (OoS) [28] metric can indicate the occlusion and crowding level of images in a dataset. This metric \(\mathrm{OoS}\) for an image of a dataset is defined as follows:  
\[\mathrm{OoS}\ =\ \left\{ \begin{array}{cc}1\ -\ \frac{\left|\cup_{i}^{n}-C_{i} \right|}{\sum_{i=1}^{C_{i}}|},&n>0\\ 0,&n\ =\ 0\end{array} \right. \tag{1}\]  
where \(C_{i}\) is the area of the bounding box (bbox) or convex hull (convex) of instance \(i\) in an image, \(n\) is the number of instances in an image, and \(\cup\) is the union operation [28]. Table 1 presents the average \(\mathrm{OoS}\) over all images of the dataset of occluded plants presented in this paper, as well as for several standard datasets. Figure 3: Examples of annotated images from the occluded plants dataset. (**a**) Normal occlusion image. (**b**) Complex occlusion image. Figure 2: Example of an image from the occluded plants dataset with normal and complex occlusion patches. The red windows illustrate some locations where normal occlusion patches are present, and the purple windows show some locations of complex occlusion patches.