Article

# Robotic Grasping of Novel Objects Based On Deep-Learning based Feature Detection

Kai Sherng Khor<fnref n="1" />, Chao Liu<fnref n="2" /> and Chien Chern Cheah <fnref n="3" />,*

**Abstract:** In recent years, the integration of deep learning into robotic grasping algorithms has led to significant advancements in this field. However, one of the challenges faced by many existing deep learning based grasping algorithms is their reliance on extensive training data, which makes them less effective when encountering novel objects not present in the training dataset. This paper presents a simple and effective grasping algorithm that addresses this challenge through the utilization of a deep learning-based object detector, focusing on oriented detection of key features shared among most objects, namely straight edges and corners. By integrating these features with information obtained through image segmentation, the proposed algorithm can logically deduce a grasping pose without being limited by the size of the training dataset. Experimental results on actual robotic grasping of novel objects over 400 trials show that the proposed method can achieve a higher grasp success rate of 98.25% compared to existing methods

**Keywords:** robotics, grasping, novel objects

## 1. Introduction

Grasping has remained a significant focus of research in robotics for decades, enabling robots to interact proficiently with objects in the real world. Humans, upon looking at an object, even for the first time, instinctively know where to pick it from. This coupled with the incredible dexterity of the human hand, makes any grasping attempt almost always a success. However, this is a highly challenging task for current robot systems as robots do not have such instinct nor dexterity [1].

Modern-day robots are highly advanced: they can be programmed to perform highly complex movements with high accuracy and precision. Yet these programs tend to be specific to a certain object or environment. A change in any of these circumstances renders these programs unreliable and a new program has to be designed. This lack of flexibility in robot grasping is a challenging problem [2][3]. The rise of deep learning algorithms has provided a possible solution to this problem [4] by introducing data-driven techniques. Through the interactions between sensors and the environment, grasp pose generations can be achieved by imitating human grasping strategies [5]. Integrating Deep Convolutional Neural Networks (CNNs) into grasping algorithms allow robots to deal with changes in the environment more readily. Modern data-driven techniques for robotic grasping involve teaching a robot with large amounts of data instead of training with physical object models [6]. This training data usually contains many images of each object in various positions and orientations, and the grasping pose is manually labelled [7][8]. Once the training is completed, the robot can grasp these objects regardless of how it is placed. Significant past works in grasping such as those by Pinto et al. [9] and Levine et al. [10] approached the problem by amassing large quantities of data over an extended period of time, 50,000

**Citation:** Khor, K.S.; Liu, C.; Cheah, C.C. Robotic Grasping of Novel Objects Based On Feature Detection. *Sensors* **2024**, *1*, 0.

Received:

Revised:

Accepted:

Published:

**Copyright:** © 2024 by the authors. Submitted to *Sensors* for possible open access publication under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

<footnote n="1">¹ School of Electrical and Electronics Engineering, Nanyang Technological University, Singapore; khor0054@e.ntu.edu.sg</footnote>

<footnote n="2">² Department of Robotics, LIRMM, CNRS-University of Montpellier, France, chao.liu@cnrs.fr</footnote>

<footnote n="3">³ School of Electrical and Electronics Engineering, Nanyang Technological University, Singapore; ECCCheah@ntu.edu.sg</footnote>

<footnote>* Correspondence: ECCCheah@ntu.edu.sg; Tel.: (65)67905385</footnote>
and 800,000 datapoints respectively. Many grasps are executed, and the results of those grasps are recorded in terms of success or failure. This allows the algorithm to learn what grasps are good. The resultant algorithm was effective in grasping specific objects. Another technique that has been explored is to use object detectors as part of the grasping algorithm. Modern object detectors are highly accurate and run considerably fast: a few iterations can run in a second, making it feasible for real-time usage. This is comparative to the early stages of CNN development whereby the high computational cost and long run times per iteration make them unsuitable for real-time detection and grasping [11]. Kim et al. [12] have trained an object detector using the images and grasping points of 2 object classes from the Open Image Dataset (OID). The resultant model can pick up these 2 classes of objects successfully with around 70% precision. Huang et al. [13] approach grasping by integrating multi-agent deep reinforcement learning with an object detector. The resultant MA-TD3H algorithm has a very high success rate in grasping objects, but can only grasp the 3 objects that it is trained on and is untested on novel objects outside of this training dataset. Some other studies find that using an object detector alone is too limited and approach the problem differently. Zhang et al. [14], Yang et al. [15], and Geng et al. [16] propose a 2-stage cascaded algorithm by first using the object detector to identify and localise an object before using a second Grasp Generation CNN (GGCNN) to determine the grasp position. A more recent study by Li et al. [17] first utilizes an object recognition algorithm to recognise 6 different categories of objects, and then develops a GGCNN that is trained with grasp positions for each category. Upon encountering an object, the object recognition module determines the category that best matches it. The GGCNN then generates a grasp position for it. The approach sports an 86% grasp success rate. However, it was noted that these systems are highly unscalable due to the limitations of the training dataset and is unable to deal with objects outside of the training. Another trade-off of cascaded algorithms are the runtimes. Geng et al. [16] found that it ran up to 4 times slower than just having the object detector alone. To deal with this limitation, multiple studies have gone beyond just robotic grasping and delved into novel object robotic grasping. This refers to objects that are not seen by the system during the training process. One of the earliest works specifically targeting novel objects grasping is by Saxena et al. [18]. In that work, 2,500 datapoints were synthetically generated to train the algorithm and it predicts a grasp pose directly as a function of the images via triangulation of a grasping point, hence removing the need to be trained on any specific objects. The algorithm has a 87.5% success rate on a collection of 8 novel objects. Mahler et al. [19] developed Dex-Net 2.0 with a synthetic dataset of 6.7 million datapoints. The algorithm showed an 80% success rate in grasping a set of 10 novel objects. Morrison et al. [20] developed a generative grasping convolutional neural network (GG-CNN) which directly generates a dense prediction of antipodal grasp poses and a quality measure for every pixel in an input depth image. They achieved a 94% success rate on a set of 12 unseen household items with a training size of 51,100 datapoints. Zhang et al. [21] tackle unknown object grasping by integrating visual information with tactile information as part of their data-driven algorithm. The trained model's performance varies greatly with the unknown object experimented with, having an accuracy of between 40% to 92%. Coupled with the large training dataset of 23,600 datapoints, the algorithm does not seem to generalise well to all novel objects. A more recent study by Eguiluz et al. [22] approaches the novel object grasping problem from a different perspective. Data-driven techniques are not used and the algorithm falls back on a more classical grasping approach using contact forces. The robot first touches the unknown object to collect more information through tactile sensors, which are combined with the RGB-D image to form a valid grasp position. As the robot is not trained via any CNN to recognise any object, all objects are novel to the robot and it has good grasp performance. However this two-step approach is slower. It requires 0.5s to generate a grasp position which is considerably slower than many other visual-based data-driven grasping algorithms that can do the same in less than 0.1s. As such, some approaches with increased complexity or combining contact forces can
improve the performance when approaching novel objects but have a significant trade-off in speed.

A common limitation in most of the existing grasping algorithms is that a large amount of training data is required but yet the success rates in actual grasping of novel objects are not high. In a world with an infinite number of different objects, it is both inefficient and impossible to train the algorithms to recognise every object. As such, a purely data-driven approach for object grasping is not ideal. In contrast to these approaches, this paper presents a different perspective in using data-driven techniques for robotic grasping by combining it with information obtained through image segmentation and logical deduction to propose a grasp pose for each novel object. Instead of training a robot to recognise objects and their grasp positions, the algorithm is trained to identify common object features, namely straight edges and corners. Combining these with some basic object information that can be extracted from the image of an object, the algorithm can logically deduce a grasp position. This algorithm does not require the tedious labelling of grasp positions on different objects, neither does it need to identify what class of object it encounters. As these are object features or object parameters that are present in almost all objects, the robot is able to attempt to grasp it even if it is not part of the training dataset and without the need to know what the object is. The algorithm achieves high performance in grasping novel objects than existing methods without the need of using large amount of training data.

## 2. Methodology

When humans grasp an object, we generally grasp it between the fingers and thumb by applying force from opposite sides. In our grasp planning, flat surfaces of the objects are preferred and corners are avoided where possible. For circular objects without flat surfaces or corners, we grasp it along the diameter. The grasping point that is chosen also tends to be near the centre of gravity of the object if possible. The centre of gravity can usually be estimated quite accurately visually as it tends to be around the centre of the object. This paper takes inspiration from these observations in human grasping to develop a robotic grasping algorithm.

### 2.1. Architecture

A 2D image X of an object is the input and it passes through two concurrent processes: features extraction and parameters extraction. The features extraction process involves identifying two key features of the object in the image, namely a set of straight edges:

$$E = \{E_1, E_2, ..., E_n\}, \qquad (1)$$

and a set of corners:

$$C = \{C_1, C_2, ..., C_m\}, \qquad (2)$$

where n is the number of straight edges detected and m is the total number of corners detected.

To achieve this, an oriented object detector is trained based on images of objects that have their edges and corners labelled instead of the class of the object. An oriented object detector is used and each detected feature is represented by a bounding box. The coordinates of the centre of a bounding box for each edge $E_i \in E$ is represented by $p_{E_i} = (x_{E_i}, y_{E_i})$. Each bounding box has a length $l_{E_i}$, a width $w_{E_i}$, and an angle $\theta_{E_i}$ where $i = 1, ..., n$. Similarly, for each detected corner $C_j \in C$ the coordinates of the centre of each bounding box is $p_{C_j} = (x_{C_j}, y_{C_j})$, and has a length $l_{C_j}$, a width $w_{C_j}$, and an angle $\theta_{C_j}$ where $j = 1, ..., m$. An illustration of the bounding box parameters is shown in Fig 1.
<figure><img src="image_1.png" /><figcaption>Figure 1. Illustration of bounding box parameters for straight edges $E_i$ and corners $C_i$.</figcaption></figure>

The object centroid ($x_c, y_c$), perimeter ($P$), and area ($A$) are also extracted. The parameters extraction process uses an image segmentation algorithm to geometrically determine the perimeter ($P_g$) of the object. From $P_g$, the enclosed area ($A_g$) can be calculated and averaging all the points on the perimeter also provides an estimated geometric centroid ($x_{cg}, y_{cg}$) of the object. Next, all the detections from the features extraction process are used to form a concave hull. This refers to the smallest geometric shape enclosing all points in a given set, which provides an abstract shape of the object and hence the estimated perimeter ($P_e$), area ($A_e$), and centroid ($x_{ce}, y_{ce}$) of the object. This abstract shape serves two important purposes. First, the parameters of the abstract shape are compared to the parameters obtained from image segmentation. From this comparison, the algorithm can determine if the features detected are a good or poor representation of the actual object. Secondly, it improves robustness of the algorithm. Should the parameters extraction fail to determine $P_g$ and hence ($x_{cg}, y_{cg}$) of the object due to unforeseen circumstances, $P_e$ and ($x_{ce}, y_{ce}$) can be used instead.

After the determination of abstract shape of the object, the algorithm then first attempts to search for a pair of parallel edges closest to the object centroid. For objects with $|E| = n \ge 2$, the algorithm pairs each edge with another to find a suitable pair closest to the centroid. The grasp position is then determined from these two edges. If no suitable parallel pair of edges can be identified, the algorithm instead identifies the longest edge as it is likely to be the most secure surface for grasping. The centre of the longest edge is then projected 90° toward the other side of the object until it intersects the perimeter. The edge and this projected point provide the grasp position. If the distance between these two points be too far apart for grasping, the algorithm then moves on to the next longest edge and repeats the process until a suitable position is found. For objects with $|E| = n = 1$ and $A$ is small, the object is likely small and thin, like a pen. The position and orientation of the edge gives the grasp position. If $|E| = n = \emptyset$, the object could be very irregular and contain only detected corners ($|C| = m \neq \emptyset$) or be a circular object with no corners at all ($|C| = m = \emptyset$). In such cases, the logic using straight edges is not possible and the algorithm employs a different strategy. A rotational search is done about the object centroid and along the perimeter for two points that give the shortest grasp distance. If $|C| \neq \emptyset$, the points enclosed by the corner's bounding box are avoided. An overview of the proposed algorithm is shown in Fig 2 and the key algorithms are presented in section 2.8.

## 2.2. Feature Learning

The basis for this algorithm lies in feature learning. By recognising common features in objects instead of specific objects, the proposed algorithm possesses the flexibility to deal with a larger variety of situations. Fig 3 shows examples of some of the data used for feature learning. Unlike other object recognition algorithms whose training data is labelled with many different classes, the feature learning in this algorithm is labelled with only two classes: edges and corners. It is important to note that the bounding boxes are oriented. It is vital for the algorithm to determine the orientation of these features as it is used in later steps of the algorithm to determine a grasp position. After the training is complete, the detector is able to identify edges and corners from an image of an object.
<figure><img src="image_2.png" /><figcaption>**Figure 2.** Overview: The input is the image of an object and it passes through two concurrent processes. The first is an oriented object detector trained on images of objects labelled with straight edges and corners which detects these same features in the image. The other step is image segmentation which extracts the parameters (perimeter, area, and centroid) of the object. An abstract shape can be derived from the detected features and compared to the actual shape to evaluate the quality of the detections. The algorithm then searches for a pair of parallel straight edges as close to the object centroid as possible. Depending on the detections, the algorithm executes a different grasp strategy.</figcaption></figure>
<figure><img src="image_3.png" /><figcaption>**Figure 3.** Labeled Data for Feature Learning: Examples of training data labelled for feature learning. Green bounding boxes represent edges and orange bounding boxes represent corners.</figcaption></figure>

In the proposed method, any oriented object detector based on deep learning can be used. Fig 4 shows the statistics of the training over 1000 epochs for 400 images using YOLOv5 OBB. The training loss decreases significantly over time and starts to plateau around 800 epochs. The precision and recall shows a high percentage of accurate predictions, with 98% correct predictions for both straight edges and corners on a clear background. These training results show the suitability of training deep learning based object detector to recognise these features and hence to develop a grasping algorithm from it.

<figure><img src="image_4.png" /><figcaption>**Figure 4.** Training losses, precision and recall of the feature learning.</figcaption></figure>

## 2.3. Abstract Shape Determination and Image Segmentation

The abstract shape determination is achieved by creating a concave hull using the centre points of all feature detections. That is, the abstract shape $S_{abstract}$ of the object is defined as:

$$S_{abstract} = Conc(P_{E} \cup P_{C}) \qquad (3)$$

where

$$P_{E} = \{p_{E_{1}}, p_{E_{2}}, \dots, p_{E_{n}}\}, \qquad (4)$$

$$P_{C} = \{p_{C_{1}}, p_{C_{2}}, \dots, p_{C_{m}}\}. \qquad (5)$$

This refers to the smallest geometrical shape formed which encloses the set of the feature points to give a general shape. Fig 5 shows the illustrations of the abstract shapes of two objects with the features determined. The abstract shape is the yellow line which gives the estimated perimeter $P_e$. The estimated area is $A_e$ and the estimated centroid $(x_{ce}, y_{ce})$ is represented by the yellow circle. If there are less than three detections such that $|E \cup C| < 3$, this step is bypassed.
<figure><img src="image_5.png" /><figcaption>**Figure 5.** Feature Detection and Determination of Abstract Shape: The red boxes show the detected straight edges and the pink boxes show the detected corners. The yellow line shows the concave hull of all these detections which provides the estimated perimeter and enclosed area of the object. The yellow circle shows the estimated centroid of the object derived from this concave hull.</figcaption></figure>

An edge detection algorithm is also used to determine the borders of the object in the image X. This is the perimeter $P_g$ which is the blue line that outlines the object as illustrated in Fig 6. The area enclosed by $P_g$ is $A_g$, and averaging all the points on $P_g$ gives the geometric centroid of the object ($x_{cg}, y_{cg}$) represented by the blue circle. Comparing $A_g$ and $A_e$ provides an idea of how closely the features represent the object. If ($x_{ce}, y_{ce}$) is also close to ($x_{cg}, y_{cg}$), it also shows its suitability in substituting ($x_{cg}, y_{cg}$) should image segmentation fail.

<figure><img src="image_6.png" /><figcaption>**Figure 6.** Image Segmentation and Parameter Extraction: Two objects with their geometric perimeter and centroid highlighted in blue.</figcaption></figure>

## 2.4. Determination of Grasp Position from Parallel Edges

With the key features and parameters of the object (straight edges, corners, perimeter, area, and centroid) extracted, the algorithm can then proceed to derive a grasp position. The algorithm determines a grasp position by identifying the two points where a two-fingered robot gripper should contact. From the features detected, the algorithm starts with a search for parallel straight edges closest to the object centroid to grasp while avoiding the corners. Fig 7(a) illustrates an example of an object with multiple detected features. For each $E_i \in E$, the distance $D_i$ from the object centroid ($x_c, y_c$) is calculated as:

$$D_i = \sqrt{(x_c - x_{E_i})^2 + (y_c - y_{E_i})^2} \qquad (6)$$

All $E_i \in E$ are then sorted in ascending order of $D_i$. Starting with $i = 1$, $E_i$ and $E_{i+1}$ are compared to find a pair that is parallel by calculating the difference in angle $\Delta\theta$ between the two edges:

$$\Delta\theta = |\theta_{E_i} - \theta_{E_{i+1}}| \le \theta_{max} \qquad (7)$$

where the initial threshold value $\theta_{max}$ is user defined. For example, an initial $\theta_{max} = 10^\circ$ can be considered as close to parallel. Fig 7(b) illustrates the comparison between two
edges. $\Delta\theta$ between these two edges is close to 90° and hence is larger than the threshold. As such this pair is not chosen.

Following this step, a series of checks ensure that the pair is properly aligned for grasping. This is done by drawing a line between the centre of the two detected edges $x_{E_i}, y_{E_i}$ and $x_{E_{i+1}}, y_{E_{i+1}}$. The acute angle $\theta_{line}$ formed by the line and one parallel edge is then calculated. The line should have at an angle difference with the angles of detections greater than a user defined threshold $\theta_{diff\_min}$:

$$|\theta_{line} - \theta_{E_i}| \geq \theta_{diff\_min} \qquad (8)$$

for instance, $\theta_{diff\_min} = 60°$. A poorly aligned pair of features as depicted in Fig 7(c) will likely be difficult to grasp. This pair is not selected as the angle difference is less than 60° from the angles of the detections and is deemed unsuitable.

The next check ensures that the distance between the 2 identified edges are not larger than the maximum opening of the gripper. This maximum opening $d_{max}$ in pixels can be calculated as such:

$$d_{max} = m \times \frac{f}{d} \qquad (9)$$

where $m$ is the measured physical maximum opening of the gripper in cm, $f$ is the focal distance of the camera in pixels, and $d$ is the measured physical distance of the camera from the workspace in cm. Fig 7(d) shows a shortlisted feature pair for grasping. Although they have a small $\Delta\theta$ and are properly aligned, the distance between them is larger than $d_{max}$ and hence is not graspable. As such, another pair with distance lesser than $d_{max}$ is chosen as illustrated in Fig 7(e). After choosing a suitable pair of edges, some optimisation is done to account for the difference in lengths of the edges detected. If the algorithm determines the grasp position from the centre of the bounding box without accounting for the differences in length, it results in a grasp position with a poor orientation as illustrated in Fig 7(e). It can be observed that the robot fingers do not have to be in the centre of the detected bounding box: it can be anywhere along the length of the bounding box. With optimisation, a grasping position that is better oriented can be generated in Fig 7(f).
<figure><img src="image_7.png" /><figcaption>**Figure 7.** An Object with Multiple Straight Edges (a) All detected features of the object. (b) Edges that have a large angle difference are not graspable. (c) Similarly, edges that are poorly aligned are poor grasp choices. To check for this, a line is drawn between 2 edges and its angle difference with the edges should be larger than $\theta_{diff\_min}$. (d) A pair can be parallel and be properly aligned, but if the distance between them is too far apart, it is not within gripper reach. (e) Another more suitable pair is chosen. If the differences in length of the bounding boxes are not accounted for and the grasp position is derived only by the middle of the bounding boxes, the grasp position is poorly oriented. (f) After optimisation, a better grasp position is achieved.</figcaption></figure>

Optimisation is achieved by projecting the centre of the shorter bounding box on the longer bounding box. Let $E_a$ represent the shorter edge of length $l_{E_a}$ with center $(x_{E_a}, y_{E_a})$ and orientation $\theta_{E_a}$, and $E_b$ represent the longer edge of length $l_{E_b}$ with center $(x_{E_b}, y_{E_b})$ and orientation $\theta_{E_b}$. With $(x_{E_b}, y_{E_b})$ and $\theta_{E_b}$, a line parallel to $E_b$ can be formed. Similarly, with $(x_{E_a}, y_{E_a})$ and $\theta_{E_a}$, a line perpendicular to $E_a$ can be formed. This line represents the projection of $(x_{E_a}, y_{E_a})$ towards $E_b$. The intersection of these 2 lines give Point K, which is the resultant point from the projection of $(x_{E_a}, y_{E_a})$ along the length of $E_b$. Hence, the 2 points of contact for the gripper are $(x_{E_a}, y_{E_a})$ and Point K to achieve an optimised grasping position. This idea of optimisation is illustrated in Fig 8.

<figure><img src="image_8.png" /><figcaption>**Figure 8.** A line parallel to longer bounding box $E_b$ and a line perpendicular to shorter bounding box $E_a$ can be formed from bounding box information. The intersection of these 2 lines at Point K give the projection of the center of $E_a$ along the length of $E_b$. $(x_{E_a}, y_{E_a})$ and Point K provide an optimised grasp position.</figcaption></figure>
It is observed that the parallel edges logic also works on non-angular shapes such as long ovals. The pointed sides of the oval are detected as corners and the elongated sides detected as edges as illustrated in Fig 9(a). Hence, the algorithm is able to generate a grasp position from these two edges in Fig 9(b).

<figure><img src="image_9.png" /><figcaption>**Figure 9.** (a) For elongated ovals, the long sides are detected as straight edges and the pointed sides are detected as corners. (b) The algorithm can generate a grasp position based on the parallel edges logic.</figcaption></figure>

## 2.5. Moving Threshold of Edges

The edges need not be perfectly parallel to be grasped. It is observed from human grasping and also robotic grasping that the edges with an angle difference of up to certain threshold can still be grasped. However, the quality and success rates of the grasp lowers with larger angles. As such, a moving threshold logic was implemented. When the algorithm is unable to find any suitable parallel pairs, it loosens the tolerance in the search of the parallel edges by allowing the selected edges to have a certain difference in angles. This tolerance is increased iteratively from an initial value (e.g. 10°) to a maximum threshold (e.g. 45°) to search for a graspable pair of edges. The step size and tolerance can be defined by users. It allows the algorithm to prioritize edges of smaller angle differences over larger ones to achieve a more stable grasp if possible. Fig 10 shows an illustration of this logic.

<figure><img src="image_10.png" /><figcaption>**Figure 10.** Object without Parallel Straight Edges: A triangle is shown with 3 detected straight edges. When the threshold is set at 0°, there is no suitable combination of parallel pairs. When this the threshold is loosened to 20°, a pair is chosen and the grasp position is established.</figcaption></figure>

## 2.6. Objects without Parallel Detected Edges

If a suitable pair of straight edges cannot be found after the loosening of constraints due to the shape of the object, the algorithm is equipped with additional logic to identify the general shape of such objects and hence execute a different grasping strategy. For an object with multiple features detected but none that fulfil the requirements for a parallel pair of edges, the algorithm identifies the longest straight edge based on $l_{E_i}$ as the point of contact for the first finger. The middle point of the edge $(x_{E_i}, y_{E_i})$ is projected at 90° towards the other side of the object and moves along the edge until it intersects the first nearest perimeter $P$ at Point $K$ with coordinates $(x_K, y_K)$. This marks the position of the second finger. The algorithm then checks the distance between these two points to ensure it is not larger than the gripper maximum opening $d_{max}$.

$$ \sqrt{(x_K - x_{E_i})^2 + (y_K - y_{E_i})^2} \le d_{max} \qquad (10) $$

If the distance is graspable, then these two points give the grasp position. Otherwise, the algorithm discards them and move on to the next longest edge to repeat the process until this condition is met. The grasp position is then established.
If $|E| = n = 1$ and A is small, the object could be a very small, stick-like object such as a spoon in Fig 11(a). The algorithm grasps at position $(x_{E_1}, y_{E_1})$ and orientation $\theta_{E_1}$ as illustrated in Fig 11(b).

<figure><img src="image_11.png" /><figcaption>**Figure 11.** (a) An object with a small area and only one straight edge is detected. (b) The algorithm grasps at that straight edge.</figcaption></figure>

## 2.7. Objects without Straight Edges

Objects without any straight edges (i.e. $|E| = \emptyset$) could be of highly irregular shapes with straight edges too small to be detected, or circular objects. As such the algorithm derives a grasp position using other available information. The shortest distance across the object centroid from perimeter to the other side of the perimeter would serve as a good and stable grasp position that is most likely to be within the grippers maximum distance. A circular object with 2 pointer corners is presented in Fig 12(a). Multiple lines $L_n$, where $n_l$ is the number of lines, are drawn from one point $A'(x_{A_{L_n}}, y_{A_{L_n}})$ along the perimeter through the object centroid to the corresponding point $B'(x_{B_{L_n}}, y_{B_{L_n}})$ on the opposite side of the object perimeter as illustrated in Fig 12(b). Corners tend to be poor grasping points as the point of contact is small, making the grasp unstable. Hence if corners are available, a check is done to ensure that none of the lines intersect the bounding boxes of the corners if possible. If they do, these lines are eliminated as illustrated by the yellow line in Fig 12. The distances $D_{L_n}$ of the remaining lines are calculated.

$$D_{L_n} = \sqrt{(x_{A_{L_n}} - x_{B_{L_n}})^2 + (y_{A_{L_n}} - y_{B_{L_n}})^2} \quad (11)$$

The shortest line is chosen, and the grasp position can be derived as illustrated in Fig 12(c).

<figure><img src="image_12.png" /><figcaption>**Figure 12.** (a) A slight oval with 2 detected corners. (b) Lines are drawn through the object centroid from one end to another at varying angles. If any of these lines intersect the bounding box of any corner, these lines are eliminated. (c) The shortest amongst the remaining lines is chosen as the grasp position.</figcaption></figure>

## 2.8. Grasp Algorithm

The following pseudo-codes summarise the approaches taken by the grasping method. Algorithm 1 shows the information extraction process from an image via feature detection and image segmentation.
**Algorithm 1 Features and parameters extraction**

1: Detect features E, C, form abstract shape

2: Calculate $P_e, A_e, (x_{ce}, y_{ce})$ from abstract shape

3: Image segmentation detects $P_g, A_g, (x_{cg}, y_{cg})$

4: **if** unable to detect **then**

5: Use $P_e, A_e, (x_{ce}, y_{ce})$

6: **else**

7: Compare $A_g$ and $A_e$ to evaluate detections

Algorithm 2 shows the steps in searching for a suitable pair of parallel straight edges closest to the object centroid.

299

300

**Algorithm 2 Search for parallel straight edges**

1: $d_{max}(px) = physicalmax(cm) \times \frac{focal(px)}{depth(cm)}$

2: **if** $n \ge 2$ **then**

3: Calculate distance $D_i$ for all $E_i$ from $(x_c, y_c)$

4: Sort $E_i$ by $D_i$ (ascending)

5: **for** $\theta_{max} = min$ to $max$ **do**

▷ user defined min/max

6: **for** $i = 1$ to $n-1$ **do**

7: **for** $j = i + 1$ to $n$ **do**

8: $\Delta\theta = |\theta_{E_i} - \theta_{E_j}|$

9: **if** $\Delta\theta \le \theta_{max}$ **then**

10: Draw line $d_i$ between $E_i, E_j$

11: Check $|\theta_{line} - \theta_i| \ge \theta_{diff\_min}$

12: Check $d_i \le d_{max}$

13: **if** suitable **then**

14: Break

15: **else**

16: $j = j + 1$

17: **if** no suitable pair **then**

18: $i = i + 1$

19: **if** no suitable pair **then**

20: $\theta_{max} = \theta_{max} + step$

▷ user defined step

21: **if** pair found **then**

22: Optimise grasp position

23: **else**

24: Sort $E_i$ by length $l_{E_i}$ (descending)

25: **for** $i = 1$ to $n$ **do**

26: $K$ = Project $(x_{E_i}, y_{E_i})$ to other side of $P$

27: Distance $K$ to $(x_{E_i}, y_{E_i}) \le d_{max}$

28: **if** suitable **then**

29: Break

30: **else**

31: $i = i + 1$

Algorithm 3 shows the steps in deducing the grasp position for objects which have no suitable parallel edges for grasping.

301

302
**Algorithm 3 No parallel straight edges**

1: **if** n == 1 and A == small **then**

2: Use $(x_{E1}, y_{E1})$ and $\theta_{E1}$

3: **else if** n == 0 **then**

4: Draw lines $L_n$ about $(x_c, y_c)$ at rotating intervals

5: Find points A' and B' intersecting P

6: **if** m ≠ 0 **then**

7: **for** i=1 to m **do**

8: Discard any $L_n$ if intersects $C_i$

9: Find $L_n$ with shortest length A'B'

10: Use points A', B' and $\theta_{A'B'}$

## 2.9. Derivation of Physical Grasp Pose

The final step before carrying out the grasp action is to convert the image coordinates into the physical coordinates. Most of the algorithm logic branches give the position of 2 points for a 2 fingered gripper to grasp. Let the coordinates of these 2 points be $(x_1, y_1)$ and $(x_2, y_2)$. The object position $(x_{obj}, y_{obj})$ and orientation $\theta_{obj}$ can be calculated from these 2 points for the robot to make the grasp attempt.

$$ (x_{obj}, y_{obj}) = \left( \frac{x_1 + x_2}{2}, \frac{y_1 + y_2}{2} \right) \qquad (12) $$

$$ \theta_{obj} = \arctan\left(\frac{x_2 - x_1}{y_2 - y_1}\right) \qquad (13) $$

Conventionally, images have their frame origin at the top left-hand corner with the x-axis pointing to the right and the y-axis downwards. However, as the camera is placed where the gripper is, its image frame origin is set at the centre of the image with its x-axis to the right, and its y-axis upwards. As the images are of a fixed resolution of w (width) by h (height) pixels, the grippers coordinates are at $(0.5w, 0.5h)$ in the image frame. As such, further adjustments have to be made to account for this frame difference as illustrated in Fig 13.

<figure><img src="image_13.png" /><figcaption>**Figure 13.** The origin of an image (green) is at the top left-hand corner whereas the origin of the gripper (blue) is at the center of the image which is at coordinates $(0.5w, 0.5h)$ in the image frame. For an object (red) in the image frame, adjustments are needed to convert it to the gripper frame.</figcaption></figure>

The coordinates of the object in the gripper frame $(x_{gripper}, y_{gripper})$ in pixels and hence real dimensions $(x_{real}, y_{real})$ in cm can then be calculated as:

$$ (x_{gripper}, y_{gripper}) = (x_{obj} - 0.5w, 0.5h - y_{obj}) \qquad (14) $$
$$ (x_{real}, y_{real}) = (x_{gripper} \times \frac{d}{f}, y_{gripper} \times \frac{d}{f}) \qquad (15) $$

where d is the depth of the camera to the object in cm, and f is the focal length of the camera in pixels.

### 3. Experiments

This section presents the physical experiments conducted to verify the efficiency of the proposed grasping algorithm. The details of the experimental set-ups are first presented, followed by the results of the experiments. The results and observations of the experiments are then discussed.

#### 3.1. Experimental Setup

In the experiments, a total of 400 images of around 100 different objects from the Cornell Grasping Dataset were used as the training data. Only the images from the dataset were used, the other grasping information was discarded. Any oriented object detector can be used, but for this study a modified version of You-Only-Look-Once version 5 (YOLOv5) algorithm called YOLOv5 OBB (Oriented Bounding Boxes) [23] was chosen as the choice of oriented object detector. Training was conducted on a Linux server equipped with a 32GB Tesla V100-DGXS Graphics Processing Unit (GPU) over 1000 epochs, taking around 10 hours to complete. The robot arm used is a Universal Robot UR5e equipped with an OnRobot RG2 Robot Gripper and the camera attached is an Intel RealSense. Both the robot and camera are connected to a PC running a 64-bit Ubuntu 18.04.5 LTS Operating System with an Intel Core i5-7500 CPU and a NVIDIA GeForce RTX 2080 SUPER GPU. A white background is used for the object to be grasped. For each object tested, 10 attempts were made. In these 10 attempts, the position and orientation of the object are varied within the field of view of the camera. A grasp is considered successful if the robot is able to grasp it and return to its ending position without dropping it in the moving process. The hardware used is shown in Fig 14.

<figure><img src="image_14.png" /><figcaption>Figure 14. From left to right: (a) Arm, (b) Gripper, and (c) Camera.</figcaption></figure>

#### 3.2. Object Test Sets

Four different sets of test objects were used. The first 3 sets of objects closely replicate the objects used in the evaluation of the following grasping algorithms in literature: Grasp Point Prediction(GPP)[18], DexNet 2.0[19], and GG-CNN[20]. This allows for a point of comparison. The fourth set of objects is a self-designed set of objects to illustrate the capabilities of the algorithm in a wide variety of scenarios. Fig 15 shows all these 4 sets of objects used for testing.
<figure><img src="image_15.png" /><figcaption>**Figure 15.** (a) Similar object set used for GPP testing [18]. (b) Similar object set used for DexNet 2.0 testing [19]. (c) Similar object set used for GG-CNN testing [20]. (d) Object set designed by this study for algorithm testing.</figcaption></figure>

Fig 15(a) shows the objects tested by GPP [18] containing 8 objects with each grasped 4 times. The objects are tape, keys, marker, toothbrush, mug, translucent box, horn, and coiled wire. Fig 15(b) shows the objects tested by DexNet 2.0 [19] containing 9 objects with each grasped 5 times. The objects are spoon, marker, toy shark, clip, whistle, tape, mouse, glue gun, and a black triangular object. Fig 15(c) shows the objects tested by GG-CNN [20] containing 12 objects with each grasped 10 times. The objects are mug, tape, clothes peg, sponge, coiled wire, marker, stuffed toy, toothbrush, screwdriver, die, ball, and rubber duck. To further illustrate the abilities of the proposed algorithm, 18 other objects as shown in Fig 15(d) were tested on top of those used by the 3 studies. These objects are stapler, scissors, glasses, flashlight, banana, bowl, canned drink, Swiss army knife, kitchen knife, circuit board, wine opener, remote control, spatula, battery pack, game console, jar, electrical component, and a slinky toy.

### 3.3. Results

The proposed method is able to generate a grasp position for each object in 0.1s. Fig 16 shows an example of the results of a typical experiment for an object. In this case, the object is a Swiss Army Knife. The figure shows how the position and orientation of the object is varied across all grasp attempts. Where possible, the shape of the object is also manipulated across attempts. In this example, the various tools of the Swiss Army Knife are opened in different combinations.
<figure><img src="image_16.png" /><figcaption>**Figure 16.** Grasp positions for 10 grasp attempts on a Swiss Army Knife with varying positions, orientations, and tool configurations.</figcaption></figure>

In total, 40 different objects were used for a total of 400 grasp attempts. Table 1 summarises the statistics for the 3 existing methods that were used for comparison. Table 2 summarises the grasp success rate of the proposed method on the 4 different object test sets, as well as the total success rate. Some objects are repeated across the 3 test sets [18][19][20] (duct tape is present in all 3 studies but is only counted once in the total success rate). Table 3 shows the full break down of all the items used and the success rate of each of them. Fig 17 shows a few examples of the physical grasp on the object by the robot after returning to its ending position and Fig 18 shows an example of a grasp position generated by the algorithm for each object tested.

<table><caption>**Table 1.** Grasp Success Rates of Existing Methods</caption><thead><tr><td>Methods</td><td>Training size</td><td>Attempts</td><td>Success rate (%)</td></tr></thead><tbody><tr><td>GPP[18]</td><td>2,500</td><td>32</td><td>87.50</td></tr><tr><td>DexNet2.0[19]</td><td>6.7 million</td><td>45</td><td>80.00</td></tr><tr><td>GG-CNN[20]</td><td>51,100</td><td>120</td><td>92.00</td></tr></tbody></table>

<table><caption>**Table 2.** Grasp Success Rates of Proposed Method on various Object Test Sets</caption><thead><tr><th>Method</th><th>Training size</th><th>Test set</th><th>Attempts</th><th>Success rate (%)</th></tr></thead><tbody><tr><td rowspan="5">Proposed method</td><td rowspan="5">400</td><td>as in GPP</td><td>80</td><td>97.50</td></tr><tr><td>as in DexNet2.0</td><td>90</td><td>97.78</td></tr><tr><td>as in GG-CNN</td><td>120</td><td>99.17</td></tr><tr><td>Self-designed</td><td>180</td><td>98.33</td></tr><tr><td>All objects</td><td>400</td><td>98.25</td></tr></tbody></table>
<table><caption>**Table 3.** Grasp success rate of each object for 10 trials each</caption><thead><tr><th>Object</th><th>Success rate(%)</th><th>Object</th><th>Success rate(%)</th></tr></thead><tbody><tr><td>Mouse</td><td>100</td><td>Kitchen knife</td><td>100</td></tr><tr><td>Duct tape</td><td>100</td><td>Marker</td><td>100</td></tr><tr><td>Canned drink</td><td>100</td><td>Toy shark</td><td>100</td></tr><tr><td>Stapler</td><td>100</td><td>Scissors</td><td>100</td></tr><tr><td>Mug</td><td>100</td><td>Rubber duck</td><td>100</td></tr><tr><td>Slinky toy</td><td>90</td><td>Keys</td><td>90</td></tr><tr><td>Stuffed toy</td><td>100</td><td>Screwdriver</td><td>100</td></tr><tr><td>Wire</td><td>90</td><td>Bowl</td><td>100</td></tr><tr><td>Sponge</td><td>100</td><td>Jar</td><td>90</td></tr><tr><td>Clothes peg</td><td>100</td><td>Circuit board</td><td>100</td></tr><tr><td>Toy Gun</td><td>90</td><td>Flashlight</td><td>100</td></tr><tr><td>Battery pack</td><td>100</td><td>Banana</td><td>100</td></tr><tr><td>Horn</td><td>100</td><td>Whistle</td><td>100</td></tr><tr><td>Spatula</td><td>100</td><td>Wine opener</td><td>100</td></tr><tr><td>Translucent box</td><td>100</td><td>Toothbrush</td><td>100</td></tr><tr><td>Swiss army knife</td><td>100</td><td>Spoon</td><td>90</td></tr><tr><td>Remote Control</td><td>100</td><td>Game console</td><td>100</td></tr><tr><td>Die</td><td>100</td><td>Triangle</td><td>100</td></tr><tr><td>Glasses</td><td>90</td><td>Ball</td><td>100</td></tr><tr><td>Electrical component</td><td>100</td><td>Clip</td><td>100</td></tr></tbody><tfoot><tr><td colspan="4">Overall success rate: 98.25%</td></tr></tfoot></table>

<figure><img src="image_17.png" /><figcaption>**Figure 17.** Successful grasps for game console, canned drink, scissors, stuffed toy, Swiss Army Knife, rubber duck, mug, wine opener, slinky toy, ball, wire, and triangle.</figcaption></figure>
<figure><img src="image_18.png" /><figcaption>**Figure 18.** Examples of grasp poses for each of the 40 objects across all test sets generated by the algorithm.</figcaption></figure>

### 3.4. Discussion

The proposed algorithm demonstrates a superior ability to grasp a large variety of unseen objects. Rectangular shaped objects like the game console (Fig 18, Object 34) are easily graspable due to its obvious straight edges. Even if they have some stray components like loose wires found in the battery pack (Fig 18, Object 23) and the electrical component (Fig 18, Object 39), the algorithm is able to ignore them and still grasp along the straight edges. Some items with smaller straight edges like the set of keys (Fig 18, Object 12) and

377

378

379

380

381

382

383
the die (Fig 18, Object 35) are also identifiable by the algorithm. It is worth noting that the algorithm works particularly well with handled objects like the kitchen knife (Fig 18, Object 2) and the spatula (Fig 18, Object 27), attributing to the property that handles are usually straight and thick which makes for very prominent parallel edges. Other odd-shaped objects like the Swiss Army Knife (Fig 18, Object 31) and the wine opener (Fig 18, Object 28) are also graspable as they have obvious parallel edges despite the overall irregular shape of the object.

The Swiss Army Knife in particular, showcases the advantage of the algorithm in terms of flexibility. Being a highly transformable object, the Swiss Army Knife was changed in shape significantly over the course of testing as illustrated in Fig 16. The algorithm aims for the middle of the handle as with Fig 19(a) where possible. When the corkscrew tool is opened at the middle of the handle, the algorithm automatically shifts its grip to the side of the Swiss Army Knife to avoid collision with the corkscrew tool in Fig 19(b).

<figure><img src="image_19.png" /><figcaption>**Figure 19.** (a) When closed or with just the main blade open, the algorithm targets the middle of the handle for grasping. (b) With the corkscrew tool open at the middle, the algorithm shifts the grasp position to the sides of the handle to avoid collision with the tool.</figcaption></figure>

The translucent box (Fig 18, Object 29) showcases the maximum gripper distance logic of the algorithm. Being a square box, it has obvious straight edges. However, this box was deliberately chosen as it has a length of about 12cm compared to the grippers maximum opening of 10cm. The algorithm is able to determine that it is too wide for grasping and hence attempts to pick the box by 1 edge instead.

The horn (Fig 18, Object 25) and the remote control (Fig 18, Object 33) are examples of objects that do not have perfectly parallel edges. However, they are still graspable due to the increasing threshold logic of the algorithm. The triangle (Fig 18, Object 36) is grasped by projecting 1 edge to the other side of the object perimeter, which happens to be a corner. With 1 edge and 1 corner in contact, the robot is able to grasp it.

Small, stick-like objects that have only 1 straight edge detected like the screwdriver (Fig 18, Object 14) and the toothbrush (Fig 18, Object 30) are grasped along this edge. For other odd-shaped objects like scissors (Fig 18, Object 8) and glasses (Fig 18, Object 37), the algorithm is also able to identify a straight edge to grasp at.

The algorithm is unable to identify any straight edges on objects like the mug (Fig 18, Object 9), the rubber duck (Fig 18, Object 10), and the ball (Fig 18, Object 38). Some have corners detected like the handle of the mug and the bill of the rubber duck, while some are circular with completely no detected features at all like the ball and duct tape (Fig 18, Object 3). The algorithm is able to derive a stable grasp about the object centroid. Avoiding the corners contributes to the stability of the grasps as well. The rubber duck had two detected corners at the beak and the tail. These would likely make for very small contact points with the gripper and result in a less stable grasp. The object which the algorithm had the most difficulty with is the toy slinky (Fig 18, Object 36). Due to the highly irregular shape of the object, the algorithm had difficulty in identifying clear features of the object. Moreover, the malleable nature of the object made it challenging for the robot to grasp as it tends to change its shape and move out of the intended grasp position. However, in spite of these challenges, the algorithm is still able to execute a stable grasp by targeting the approximate
centroid of the object. This allows the object to be more balanced and less likely to slide out of the gripper. Hence, the algorithm still sports a high success rate in grasping the toy slinky, further showing its flexibility in dealing with challenging situations.

The algorithm sports a significantly higher grasp success rate compared to other existing methods on a significantly smaller training dataset. This can be attributed to the greatest advantage of this algorithm which is flexibility. In other existing methods, when encountering an object, the algorithm finds the closest match based on what it is trained on. However, as illustrated in Fig 16, even within a single object there can be many orientations, positions, and configurations. This makes it impossible to train an algorithm to be able to deal with every possible combination. In contrast, the features of an object do not change as easily. Even when changing configurations the key features of the object remain unchanged. The algorithm is equipped to re-actively deduce a grasp position based on the features presented, providing it with a high level of flexibility. Unlike other methods where the grasp position of an object in every orientation and configuration has to be labelled, this study has demonstrated that with feature detection, this tedious labelling process is not required and the algorithm has the flexibility to handle a large variety of cases.

## 4. Conclusion

In this paper, a simple and effective algorithm has been proposed to enable robots to grasp novel objects. Unlike most approaches which focused on recognizing specific objects or predetermined grasp positions, this algorithm is trained to recognize common key features present in most objects, such as straight edges and corners. Leveraging these identified features and information obtained from segmentation, the algorithm can deduce information about the object and derive suitable grasp positions. Experimental results show that the algorithm is capable in grasping a huge variety of novel objects, achieving an higher success rate of 98.25 % than existing methods.
References

1. A. Bicchi, "Hands for Dexterous Manipulation and Robust Grasping: A Difficult Road Towards Simplicity," IEEE Transactions on Robotics and Automation, vol. 16, no. 6, pp. 652-662, 2000.

2. H. Zhang, J. Tang, S. Sun, X. Lan, "Robotic grasping from classical to modern: A survey," arXiv preprint arXiv:2202.03631, 2022.

3. J. Souza, L. Rocha, P. Oliveira, A. Moreira, J. Boaventura-Cunha, "Robotic grasping: from wrench space heuristics to deep learning policies," Robotics and Computer-Integrated Manufacturing 71, 102176, 2021.

4. Y. LeCun, Y. Bengio, G. Hinton, "Deep learning," nature, 521(7553), pp.436-444, 2015.

5. H. Tian, K. Song, S. Li, S. Ma, J. Xu, Y. Yan, "Data-driven robotic visual grasping detection for unknown objects: A problem-oriented review," Expert Systems with Applications, 211, p.118624, 2023.

6. K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image recognition," Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770-778), 2016.

7. A. Krizhevsky, I. Sutskever, G. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems, 25, 2012.

8. J. Bohg, A. Morales, T. Asfour, D. Kragic, "Data-driven grasp synthesis—a survey," IEEE Transactions on Robotics, vol. 30, pp. 289-309, 2013.

9. L. Pinto, A. Gupta, "Supersizing Self-supervision: Learning to Grasp from 50K Tries and 700 Robot Hours," IEEE International Conference on Robotics and Automation (ICRA), pp. 3406-3413, 2016.

10. S. Levine, P. Pastor, A. Krizhevsky, D. Quillen, "Learning hand-eye coordination for robotic grasping with deep learning and large scale data collection," International Symposium on Experimental Robotics (ISER), 2016.

11. P. Adarsh, P. Rathi, "YOLO v3-Tiny: Object Detection and Recognition using one stage improved model," International Conference on Advanced Computing & Communication Systems (ICACCS), vol. 6, pp. 687-694, 2020.

12. K. Kim, S. Kim, "YOLO-based robotic grasping," The 21st International Conference on Control, Automation, and Systems, pp. 1120-1122, 2021.

13. Y. Huang, D. Liu, Z. Liu, K. Wang, Q. Wang, J. Tan, "A novel robotic grasping method for moving objects based on multi-agent deep reinforcement learning," Robotics and Computer-Integrated Manufacturing, 86, 102644, 2024.

14. S. Zhang, Z. Guo, J. Huang, W. Ren, L. Xia, "Robotic Grasping Position of Irregular Object Based YOLO Algorithm," 5th International Conference on Automation, Control, and Robotics Engineering (CACRE), pp. 642-646, 2020.

15. J. Yang, U. Chen, K. Chang, Y. Chen, "A Novel Robotic Grasp Detection Technique by Intergrating YOLO and Grasp Detection Deep Neural Networks," International Conference on Advanced Robotics and Intelligent Systems (ARIS), vol. 1, 2020.

16. Z. Geng, G. Chen, "A Novel Real-time Grasping Method Cobimbed with YOLO and GDFCN," IEEE 10th Joint International Information Technology and Artificial Intelligence Conference (ITAIC), pp. 500-505, 2022.

17. Z. Li, B. Xu, D. Wu, K. Zhao, S. Chen, M. Lu, J. Cong, "A YOLO-GGCNN based grasping framework for mobile robots in unknown environments," Expert Systems with Applications, 225, p.119993, 2023.

18. A. Saxena, J. Driemeyer, J. Kearns and A. Ng, "Robotic grasping of novel objects," Advances in neural information processing systems, vol. 19, 2006.

19. J. Mahler, J. Liang, S. Niyaz, M. Laskey, R. Doan, X. Liu, J. A. Ojea and K. Goldberg, "DexNet 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics," arXiv preprint arXiv:1703.09312, 2017.

20. D. Morrison, P. Corke and J. Leitner, "Learning robust, real-time, reactive robotic grasping," The International Journal of Robotics Research, vol. 39, no. 2-3, pp. 183-201, 2020.

21. Z. Zhang, Z. Zhang, L. Wang, X. Zhu, H. Huang, Q. Cao, "Digital twin enabled grasp outcomes assessment for unknown objects using visual tactile fusion perception," Robotics and Computer-Integrated Manufacturing, 84, 102601, 2023.

22. A. Eguíluz, I. Rañó, "Heuristic grasping of convex objects using 3D imaging and tactile sensing in uncalibrated grasping scenarios," Expert Systems with Applications, 208, p.118114, 2022.

23. M. Kelly, "YOLOv5 for Oriented Object Detection," roboflow, [Online]. Available: https://blog.roboflow.com/yolov5-for-oriented-object-detection/, 11 May 2022.

**Disclaimer/Publisher's Note:** The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.