# Robotic Grasping of Unknown Objects Based on Deep-Learning Based Feature Detection
## 2 Methodology
### Architecture  
A 2D image \(X\) of an object is the input and it passes through two concurrent processes: _feature extraction_ and _parameter extraction_. The feature extraction process involves identifying two key features of the object in the image, namely a set of straight edges:  
\[E=\{E_{1},E_{2},...,E_{n}\}, \tag{1}\]  
and a set of corners:  
\[C=\{C_{1},C_{2},...,C_{m}\}, \tag{2}\]  
where \(n\) is the number of straight edges detected and \(m\) is the total number of corners detected. To achieve this, an oriented object detector is trained based on images of objects that have their edges and corners labelled instead of the class of the object. An oriented object detector is used and each detected feature is represented by a bounding box. The coordinates of the center of a bounding box for each edge \(E_{i}\in E\) are represented by \(p_{E_{i}}=(x_{E_{i}},y_{E_{i}})\). Each bounding box has a length \(l_{E_{i}}\), a width \(w_{E_{i}}\), and an angle \(\theta_{E_{i}}\), where \(i=1,...,n\). Similarly, for each detected corner \(C_{j}\in C\), the coordinates of the center of each bounding box are \(p_{C_{j}}=(x_{C_{j}},y_{C_{j}})\), and it has a length \(l_{C_{j}}\), a width \(w_{C_{j}}\), and an angle \(\theta_{C_{j}}\), where \(j=1,...,m\). An illustration of the bounding box parameters is shown in Figure 1.  
The object centroid \((x_{c},y_{c})\), perimeter (\(P\)), and area (\(A\)) are also extracted. The parameter extraction process first uses an image segmentation algorithm to geometrically determine the perimeter (\(P_{g}\)) of the object. The area which is enclosed by the perimeter is then determined as \((A_{g})\). The average of all the points on the perimeter also provides an estimated geometric centroid \((x_{cg},y_{cg})\) of the object. An illustration is shown in Figure 2.  
Next, all the detections from the feature extraction process are used to form a concave hull. This refers to the smallest geometric shape enclosing all points in a given set, which provides an abstract shape of the object and hence the estimated perimeter (\(P_{e}\)), area (\(A_{e}\)), and centroid \((x_{ce},y_{ce})\) of the object. This abstract shape serves two important purposes. First, the parameters of the abstract shape are compared to the parameters obtained from image segmentation. From this comparison, the algorithm can determine if the features detected are a good or poor representation of the actual object. Secondly, it improves robustness of the algorithm. Should the parameter extraction fail to determine \(P_{g}\) and hence \((x_{cg},y_{cg})\) of the object due to unforeseen circumstances, \(P_{e}\) and \((x_{ce},y_{ce})\) can be used instead. After the determination of the abstract shape of the object, the algorithm then first attempts to search for a pair of parallel edges closest to the object centroid. For objects with \(|E|=n\geq 2\), the algorithm pairs each edge with another to find a suitable pair closest to the centroid. The grasp position is then determined from these two edges. If no suitable parallel pair of edges can be identified, the algorithm instead identifies the longest edge, as it is likely to be the most secure surface for grasping. The center of the longest edge is then projected 90\({}^{\circ}\) toward the other side of the object until it intersects the perimeter. The edge and this projected point provide the grasp position. If the distance between these two points is too far apart for grasping, the algorithm then moves on to the next longest edge and repeats the process until a suitable position is found. For objects with \(|E|=n=1\) and small \(A\), the object is likely small and thin, like a pen. The position and orientation of the edge gives the grasp position. If \(|E|=n=\emptyset\), the object could be very irregular and contain only detected corners \((|C|=m\
eq\emptyset)\) or be a circular object with no corners at all \((|C|=m=\emptyset)\). In such cases, the logic using straight edges is not possible and the algorithm employs a different strategy.