# Accurate Instance-Based Segmentation for Boundary Detection in Robot Grasping Application
## 3 Experimental Methodology
### Proposed Method  
We propose a method that combines Mask R-CNN and 3D segmentation algorithms, which are Difference of Normals (DoN)-Based Segmentation and Euclidean Cluster Extraction. First, we add a 3D process branch to save all pixels of objects and edge regions, which are usually treated as background if just considering only 2D features. Then, the distance between these edges and central regions is submitted to consider the relationship between edge regions and object class. In robot grasping, objects are usually disorganized, scalar, close together, overlapping, and obscured by each other. These things make an enormous hurdle for Mask R-CNN on reaching the high accurate segmentation. Taking advantage of the 3D information, 3D segmentation algorithms are applied to deal with this barrier and enhance the accuracy of Mask R-CNN. Certainly, Figure 2 depicts our detailed works. The first step is rejecting almost the background to reduce the computation cost, since objects are usually located in a specified region that is limited by the workspace of the robot. Thus, based on the position of objects in space, objects out of the workspace of robots could be easily denied. This step dismissed most of the background and significantly decreased computation on the following segmentation steps. After generating the point cloud from the depth map acquired from Kinect V1 [24], the plane in which the objects are arranged is dismissed by random sample consensus (RANSAC) [25]. The numbers of the point cloud decreased dramatically, and the cost of computation also significantly decreased. Figure 1: Diagram of combination between original Mask R-CNN and 3D segmentation algorithms. A point cloud process branch goes together with Mask R-CNN to operate 3D segmentation algorithms. The result after applying our proposed method to objects in full. The second step is applying DoN for the rough segmentation. Unlike Grabcut, DoN only depends on the Difference of Normals of points but does not depend on the color of the image. Thus, it does not miss any region even though the edge region. DoN compares, at each point \(\mathbf{p}\), the responses of the operator across two different radii \(r_{s}<r_{l}\). Figure 3 illustrates the effect of the support radius on the estimated surface normal for a point cloud. Formally, the Difference of Normals operator of any point \(\mathbf{p}\) in a point cloud is defined as follows [10,26]. \[\Delta_{\hat{n}}(p,r_{s}r_{l})=\frac{\hat{n}(p,r_{s})-\hat{n}(p,r_{l})}{2} \tag{1}\]  
where \(r_{l},r_{s}\in\mathbb{R}\), \(r_{s}<r_{l}\), and \(\hat{n}(p,r)\) is the surface normal estimate at point \(\mathbf{p}\), given the support radius \(r\). DoN is implemented with the support of PCL following these steps for segmentation [26]:  
1. Estimate the normals for every point using a large support radius of \(r_{l}\). 2. Estimate the normals for every point using the small support radius of \(r_{s}\). 3. For every point, the normalized Difference of Normals for every point, as defined above. 4. Filter the resulting vector field to isolate the points belonging to the scale/region of interest. Figure 3 confirms that Grabcut segmentation does not work more effectively than Difference of Normals. Practice in a situation as described in Figure 3a, Mask R-CNN gains  
Figure 2: Flow chart of our approach for improving edge region segmentation. an unimpressive result as shown in Figure 3b. Expected to improve that result, however, Grabcut is limited by the low resolution and the contrast of input image taken from a 3D camera. Whereas, DoN can roughly separate the object without forgetting any edge regions. Compared with the combined Mask R-CNN and Grabcut method proposed by [PERSON] as Figure 3c describes, Grabcut denies the regions near the border area, while DoN, as in Figure 3d, can save more misclassified pixels. The third step is distinguishing the border region and the missing parts that object Mask R-CNN does not cover.