# Self-Recognition Grasping Operation with a Vision-Based Redundant Manipulator System
## 4 Simulation and Experiments
### Experiment on Finding and Locating Target Object  
For the self-recognized grasping operation, the primary step is to the distinguish target object from the unstructured environment. Here, we set some fruit on the shelf shown in Figure 5a. There are two kinds of fruit (an apple and orange), and two apples were placed on different layers. We would like the manipulator to distinguish the three objects and grasp the upper apple by itself.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Number** & \(\mathbf{\alpha}(^{\circ})\) & \(\mathbf{a}(\mathbf{mm})\) & \(\mathbf{\theta}(^{\circ})\) & \(\mathbf{d}(\mathbf{mm})\) \\ \hline
1 & 90 & 0 & \(-90\) & \(d_{1}=380\) \\
2 & \(-90\) & 0 & 0 & 0 \\
3 & 90 & 0 & 0 & \(d_{2}=328\) \\
4 & \(-90\) & 0 & 0 & 0 \\
5 & 90 & 0 & 0 & \(d_{3}=323\) \\
6 & \(-90\) & 0 & 0 & 0 \\
7 & 0 & 0 & 90 & 0 \\ T & 0 & 0 & 0 & \(d_{4}=190\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: DH (Denavit-Hartenberg) parameters of 7-DOF serial manipulator.  
Figure 4: (**a**) The joint coordinates of 7-DOF redundant manipulator, (**b**) the configuration of the 7-DOF redundant manipulator.  
The experimental scene obtained via a camera is shown in Figure 5b. The shelf was set as an obstacle and limitation to the workspace of the robot in its grasping operation. Different objects belonging to the same category (apple) were set to confuse the recognition of the target object. The background is a common lab, which is considered as clutter, confusing the feature detection and pose estimation. The experimental scene can be treated as a typical unstructured environment.  
The instance segmentation algorithm is devoted to mark the object and provide a result, as shown in Figure 5c. The fruit on the shelf can be clearly marked by category, and the two apples are also distinguished with different colors. The confidence coefficient of each object is labeled in the figure, and the target apple had the highest confidence coefficient, which was 0.90. Besides, the confused background can be almost eliminated via instance segmentation. As a result, the target apple is clearly recognized for further operation.  
After finding the target apple, the position was detected as the center of the contour shown in Figure 5c. The orientation for grasping was treated as a classification problem, based on the grasping network described in Section 2. The contour was divided into 36 parts around the circumference, where each part is equal to a 10\({}^{\circ}\) angle, turning the orientation estimation into a 36-way binary classification problem. The orientation estimation result is shown in Figure 5d. The orientation angle of the target apple was classified as 10\({}^{\circ}\).  
By taking a photo via the vision system, the manipulator can find the target object from the unstructured environment and obtain the contour of the target object, then obtain the position and orientation angle autonomously. According to the experiment, the efficiency of instance segmentation and grasping network is verified via self-recognized grasping. The benefit from instance segmentation and grasping orientation estimation was used, concerning the contour of the target object. The contour marked by instance segmentation is more accurate than bounding box classification, which greatly reduces the range and time cost for the grasping network to estimate the grasping orientation.  
Figure 5: Instance segmentation and pose estimation based on a ZED camera set on the spatial redundant manipulator. (**a**) The experiment scene of grasping, (**b**) image obtained by ZED camera, (**c**) the result of instance segmentation, (**d**) and the result of pose estimation.