# Deep Learning-Based Occlusion Handling of Overlapped Plants for Robotic Grasping
## 3 Materials and Methods
### Grasp Detection
#### 3.4.3. Optimal Grasp Detection  
To find the optimal grasp among the grasp proposals, in the first stage, the unstable grasps are filtered out, and in the second stage the orientation of the remaining grasp  
Figure 7: RoIs segmentation pipeline. (**a**) Segmented plant cutting extracted from an original RGB image. (**b**) Segmented plant cutting’s mask after preprocessing. (**c**) Convex hull mask. (**d**) RoIs mask where each color represents one RoI.  
proposals is refined, then the grasps are evaluated against a defined optimality measure to find the optimal grasp.  
To filter out the most likely unstable grasps, the following two characteristics of plant cuttings are considered. First, the distance between the two points of each proposed grasp should be within an acceptable range determined by the minimum and maximum possible width of the cutting's stem in the pixel unit. Second, the so-called \"Cutting's Interested Region\" (CIR) is defined as a region in the bounding box of a plant cutting where all possible grasps should be placed in. Furthermore, CIR is a portion of the bounding box of the plant cutting, and its center is aligned with the center of the bounding box. Since the plant cuttings are oriented vertically and the stems are almost in the middle of the bounding boxes of the cuttings, the horizontal ratio of the CIR and the bounding box of the cuttings should be much smaller than the vertical one. The set ratios between the CIR and the bounding box of the cuttings for horizontal and vertical sides are 0.5 and 0.9, respectively.  
After applying the above constraints to the grasp proposals, the bulk of the unstable grasps are filtered out. In the next step, the orientation of the remaining grasp proposals is refined to be aligned with the direction of the plant cutting's stem. Since the plant cutting's stem is roughly cylindrical, the orientation of a grasp proposal would be aligned along the stem if the distance between the points of the grasp is approximately as large as the width of the stem. To refine the orientation of a grasp proposal, one point of the grasp remains fixed and another is moved along the stem to minimize a function that calculates the distance between two points of a grasp proposal. The remaining grasp proposals are refined based on this method to increase the precision of the grasps' orientation.  
Finally, for finding the optimal grasp, we defined a metric called Overlapped Grasp Index (OGI). This metric uses both the pair of points and the rectangle representations of grasps.  
\[\text{OGI }=\frac{Grasp\;rectangle\;\cap\;Plant\;Cutting\;mask}{(Distance\; \;of\;the\;points\;of\;the\;grasp\;)\;\times\;(Grasp\;height)} \tag{3}\]  
OGI indicates the overlap of the grasp rectangle with the leaves of the cutting. The minimum, indeed the optimal value of this metric, is 1, and the higher values show the overlap of the grasp with the cutting's leaves. Thus, the proposed grasp with the smallest OGI is the optimal local grasp for the cutting. Figure 8 shows the key intermediate results of the grasp proposal and optimal grasp detection pipeline.  
Figure 8: Key intermediate results of the grasp proposal and optimal grasp detection pipeline. (**a**) All grasp proposals are marked as green points on the boundaries of ROIs. In principle, any pair of the points from two separate RoIs is a proposed grasp (**b**) Grasp proposal after filtering out the most likely unstable grasps. (**c**) Refined proposed grasps. Grasps marked with red rectangles are excluded due to overlap with leaves as determined by high OGI. (**d**) Local optimal grasp, which has the smallest OGI.