# Semantic 3D Reconstruction for Robotic Manipulators with an Eye-In-Hand Vision System
## 4 Experimental Results
### Three-Dimensional Reconstruction Results  
In order to prove the accuracy of the algorithm, we tested the following four methods:  
1. Mask-only: mask R-CNN for image segmentation and the forward kinematics for camera position calculation;
2. Mask+ICP: mask R-CNN for image segmentation, the forward kinematics, and ICP registration for camera position calculation;
3. SLIC+ICP: Simple Linear Iterative Cluster (SLIC), the forward kinematics, and ICP registration for camera position calculation; and
4. Our method: mask R-CNN is combined with the LCCCP method for image segmentation, the forward kinematics, and SICP registration for camera position calculation. After building the 3D model of five types of objects with these methods, we made the ICP match with the ground that was true of the object to calculate the object reconstruction accuracy and the Cloud to Cloud (C2C) absolute distance. The results are shown in Figure 4. Figure 4a shows the original image, and Figure 4b-e represent the 3D reconstruction results of mask-only, mask + ICP, SLIC + ICP, and our method, respectively, with different color points representing different types of objects. Table 3 shows the Cloud to Cloud (C2C) absolute distance between object models and 3D reconstruction by four methods. The higher value of C2C absolute distance means the lower precision of the 3D reconstruction. The comparison results show that as the camera position is inaccurate due to \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Method** & **Mean** & **Cylinder** & **Half-Cylinder** & **L-Block** & **Cuboid** & **Box** \\ \hline Mask R-CNN [40] & 90.782 & 92.128 & 90.168 & 92.788 & 86.498 & 92.327 \\ Our method & 92.958 & 92.174 & 91.330 & 92.991 & 92.443 & 95.852 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Our dataset results on instance segmentation Intersection-over-Union (IoU) (%). \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Method** & **Mean** & **Cylinder** & **Half-Cylinder** & **L-Block** & **Cuboid** & **Box** \\ \hline Mask R-CNN [40] & 90.782 & 92.128 & 90.168 & 92.788 & 86.498 & 92.327 \\ Our method & 92.958 & 92.174 & 91.330 & 92.991 & 92.443 & 95.852 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: YCB dataset results on instance segmentation IoU (%). the robotic manipulator motion error, the mask-only method has the lowest modeling accuracy, and the image of each frame does not overlap well. Since the mask + ICP and the SLIC + ICP method optimizes the camera position, the image coincides well, and the model accuracy is greatly improved compared to the non-optimization method. Our method improves the accuracy of 3D reconstruction based on the mask + ICP method because it improves the segmentation quality of each frame of image. We evaluated the accuracy of 3D reconstruction by the method introduced in [43]. The 3D reconstruction accuracy of the four methods on our dataset is shown in Figure 5. Due to the poor quality of the image segmentation boundary of mask R-CNN, the reconstructed model has a large number of misidentification points. The motion error of the robotic manipulator and the camera calibration error result in an inaccurate position of the camera in the world coordinate system, so, when directly using the mask R-CNN method, the 3D modeling accuracy is low, with only 28.18% of the points in 1 mm distance to the model. Since the mask + ICP method optimizes the camera position, the 3D modeling accuracy is improved compared to the non-optimization method, but as the quality of image segmentation is still poor, only 48.23% of the points are within 1 mm of the model. Our method employs fusion segmentation to improve the quality of image segmentation and also uses the segmentation SICP method to finely correct the image, so it has the highest modeling accuracy among the four methods, and the average precision reaches 53.16%, which is improved by 25.49% over the mask-only method, 4.93% over the mask + ICP method, and 3.50% over the SLIC+ICP method. Figure 4: Comparison of the semantic map with four methods.