# Design of an Eye-in-Hand Smart Gripper for Visual and Mechanical Adaptation in Grasping
## 5 Visual Adaptivity by Eye-in-Hand System
### Finding Object's Location, Orientation and Shape  
After identifying the target object in the captured image, the Douglas-Peucker [20] algorithm was used to find the vertices of the contour of the object. Using the Douglas-Peucker algorithm, a polyline composed of numerous points can be simplified into a few vertices. As an example, as shown in Figure 21a, the original polyline contains eight points that start from point A to point B. Then, in Figure 21b, when drawing line \(a\), which links from point A to point B, we can find point P\({}_{1}\), which is the furthest point to line \(a\). If the distance \(b\) between line \(a\) and P\({}_{1}\) is larger than the threshold value set for the Douglas-Peucker algorithm, P\({}_{1}\) is one of the vertices in this polyline, and the polyline can be separated into two. Using the same rule, P\({}_{2}\) and P\({}_{3}\) are shown in Figure 21c,d, respectively. Finally, we obtained a new polyline constructed with five points and three vertices, as shown in Figure 21e. Figure 19: Flowchart depicting the grasping operation of the system. Figure 20: (**a**) Original RGB image of the metal block object on a green-color conveyer belt and (**b**) after being processed by the HSV color model. The shape of the target object is determined by calculating the number of vertices. Utilizing the locations of these vertices, the orientation and centroid location of the target object can be determined by using trigonometric functions. As illustrated, examples as shown in Figure 22, the object contour and corresponding centroid location (marked by red dots) were found by using the Douglas-Peucker algorithm. With the found contour, the objects were further classified into triangular, rectangular, or circular shapes according to the number of edge points, which is the number of vertices of the contour of the object. When three edges were detected, the target was determined as a triangle. If the number of vertices of a target exceeded the threshold, set to 10 in this case, it was considered as a circular object in this study. As indicated in the left portion of Figure 22, \(\alpha\), which is defined as the angle between the moving axis of the underlying conveyer belt and the edge of the object, can be calculated to provide orientation information of the target object with respect to the fixed moving direction of the conveyor belt. With the Douglas-Peucker algorithm applied to the captured image from the eye-in-hand camera, the adaptive gripper can find each edge point's location, as well as the orientation and location of the target object by calculating the edge  
Figure 21: Illustration of using the Douglas–Peucker algorithm on (**a**) captured points of an object’s edge contour curve, and progression to find P\({}_{1}\), P\({}_{2}\), and P\({}_{3}\) edge points in (**b–d**), respectively, to form (**e**) final polyline showing three vertices of the contour. Figure 22: Shape profile and centroid location identified by the Douglas–Peucker algorithm (**left**). Illustration of two-finger (**right-top**) and three-finger (**right-bottom**) modes of the gripper for the corresponding object shapes. points. With the shape, location, and orientation data, the gripper can easily locate the target object and adjust its finger orientations to match the object in grasping operations. As the camera coordinates and gripper coordinates coincided, calibration between the coordinates was not necessary, which facilitated the control practice of the gripper to grasp targets in engineering applications. As shown on the right side of Figure 22, the target object should have a triangular or circular shape, and a three-finger mode is engaged, in which each of the gripper fingers is oriented 120\({}^{\circ}\) with each other for an equilateral triangle or a circular object. If the triangle-shaped object is not an equilateral triangle, the three-finger mode is still employed, but the orientation of each gripper finger is automatically adjusted accordingly.