# Deep Learning-Based Occlusion Handling of Overlapped Plants for Robotic Grasping
## 4 Results and Discussion  
### Occlusion Handling  
The occluded plant dataset was split into sets of 500, 50, and 100 real images for training, validation, and testing, respectively. The test dataset itself included 50 normal occlusion images and 50 complex occlusion images. In addition to the real images, we included 5000 synthetic images in the training set. AdaptIS was trained through a two-stage training procedure. The backbone and segmentation head were first trained for 230 epochs, then these parts were frozen, and only the point proposal head was trained for additional 15 epochs. The first stage consisted of 220 epochs of training with real and synthetic images, followed by 10 epochs of fine-tuning training with real images. In the second stage of the training procedure, only the real images were used. For the backbone, we used ResNet50 [35] and trained the network using Adaptive Moment Estimation (Adam) Optimizer with 2 GPUs and batch size 4. At both training stages, the base learning rate was 0.0005 and it was reduced by the cosine learning rate scheduler. Each original image contained a region of interest with a resolution of 1536 \(\times\) 1536 pixels, such that all cutting instances were always placed in this region of interest. The input size of the network was 512 pixels, thus, downscaling the original image from 1536 to 512 pixels resulted in a drastic loss of information. To reduce this downscaling in the training step, first, the part of the image where the plants were placed was cropped out and then scaled down to 512 pixels. An additional advantage of this method is that it acts as a random scaling augmentation in the range of (0.33 to 0.66). Apart from scale augmentation, we also used random rotation, brightness, contrast, and color data augmentation. Table 2 presents the panoptic segmentation result of AdaptIS for normal and complex occlusion. The standard metrics of panoptic segmentation are Panoptic Quality (PQ), Segmentation Quality (SQ), and Recognition Quality (RQ) [31]. Each of the panoptic metrics is split into metrics for the Stuff (PQ\({}^{\text{St}}\), SQ\({}^{\text{St}}\), RQ\({}^{\text{St}}\)) and Thing classes (PQ\({}^{\text{Th}}\), SQ\({}^{\text{Th}}\), RQ\({}^{\text{Th}}\)) [31]. In the presented work, only the Image Background represented a Stuff class, and the results for that were about 100 on all metrics, so we present here the panoptic segmentation result only for Thing classes to show more clearly the performance of AdaptIS on the occluded plants dataset. The optimal input size of the network in inference was 640 pixels. The inference time was 650 ms per image on the used local machine, with a single RTX 2080 Ti GPU, MXNET 1.7, and CUDA 10.2. Surprisingly, as the results in Table 2 show, the accuracy of AdaptIS for Occluded and Target Cutting classes is very close for the normal and complex occlusion, confirming the robustness of AdaptIS in segmenting severe occluded scenes. As expected, the Occluded Cutting instances are the most challenging class for segmentation, and the accuracy of the network is lower for them than for other classes. Figure 9 shows examples of panoptic segmentation results with AdaptIS for normal and complex occlusion. It can be seen that the Occluded Cutting instances in almost all examples are split into multiple parts, or the large portion of a cutting in the example at the bottom left is hidden. Although these are difficult cases for a segmentation task, AdaptIS successfully handled them and accurately segmented these Occluded Cuttings. The network moreover makes a precise distinction between the Target cutting and Occluded cuttings in the cases of complex occlusions.