# Target Recognition and Grabbing Positioning Method Based on Convolutional Neural Network

Authors: Feng Mei; Xingyu Gao; Shichao Deng; Weiming Li
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

With the continuous reform of intelligent manufacturing, industrial production has gradually developed from automation to intelligence. The fusion of vision technology and industrial machines has become a hot research direction in current intelligent transformation. However, machines are not as flexible as humans when grabbing, and still have great limitations. Affected by various characteristics of target objects, such as shape, material, weight and other factors, as well as complex and changeable environmental factors, the research of machine grabbing still faces severe challenges. For the actual complex working conditions, the poor target detection effect leads to the inability to complete accurate grabbing, which affects the production efficiency. This paper proposes a grabbing system with convolutional neural network, which can achieve target detection, classification, positioning and grabbing tasks. First, by comparing the current mainstream target recognition and detection algorithms, select SSD that have both real-time performance and accuracy. Then make specific network structure improvements according to the detection requirements, and insert the Inception structure. At the same time optimize its loss function and nonmaximum suppression. The improved recognition rate is higher, and the target detection frame is closer to the real part, which greatly reduces the recognition error. Second, this research proposes an algorithm model for regional posture detection and grabbing positioning, which uses the output of the previous stage as input to perform posture detection and grabbing positioning of the grabbed target. In the network, the posture angle of the grabbing target is output in a classified manner, and the position coordinates of the grabbing point are output using a regression method. Experiments have proved that our method can perform efficient target recognition and grabbing positioning.


---

## Body

## 1. Introduction

Industrial machines have been widely used in automobile manufacturing, mechanical processing, electronic and electrical, food processing, logistics and other industries. Since the turn of the twenty-first century, robotics has advanced at an exponential rate, and new technologies are being implemented in manufacturing on a regular basis. A country’s level of automation and technical advancement can be gauged by the development of machines. Automation systems must become increasingly intelligent, information-based, high-precision, and high-reliability as the economy grows, production scales expand, and the complexity of the production environment increases. Machine vision is a key technology for robots in the fields of industrial detection, target recognition, workpiece positioning, object tracking, and robot navigation. It has the advantages of noncontact, high precision, flexibility and reliability, and good environmental adaptability. Machine vision uses a camera to collect scene images, and through image processing and analysis, obtains the characteristics and location information of the target, and thus controls the robot to operate. This process can be widely used in fields such as automatic identification of assembly line workpieces, nondestructive testing of parts, package sorting, and welding seam measurement. This can effectively solve the shortcomings of human detection labor intensity, low efficiency, and traditional teaching-type industrial robots’ strong environmental dependence, single function, and low degree of automation [1–5].A new branch in artificial intelligence study, deep learning, has been a long-standing focus of human research. The process of learning is analogous to the process of acquiring knowledge. The traits or features are identified through the ongoing abstraction of images and other data. There has been tremendous progress in object recognition, detection, and tracking over the past few years thanks to the advent of deep learning technologies. Many deep learning models have already attained real-time processing speeds as a result of the development of GPUs and other high-performance hardware facilities. In the subject of robotics, the use of deep learning in industrial production is now a research focus. Object detection is a popular study topic in the realm of conventional vision. And in recent years, deep learning has become a hot topic in the field of target recognition and localization. Convolutional neural networks in deep learning, in particular, have a direct impact on target recognition and localization. The top international computer vision conferences have continuously proposed new deep learning network models, and constantly refreshed the accuracy of various large-scale image competitions. R-CNN series networks based on region selection and End-to-End Yolo, SSD and other networks have made great progress in speed and accuracy, and some networks have been able to achieve real-time processing capabilities [6–10].It is critical to the success of industrial operations to be able to identify and locate targets. It is necessary for many manufacturing businesses to identify, classify and detect items. The degree of automation and intelligence of the workshop is closely related to industrial production efficiency, especially for companies with large production scales, a variety of production types, and a variety of production conditions. Automate industrial processes by incorporating machine vision and deep learning into robots. This will significantly increase production efficiency, cut industrial production costs, and improve industrial management and information technology [11–15].This topic studies the target recognition and grabbing positioning, and applies the image processing algorithm and target detection algorithm with convolutional neural network to the system, so that robots can intelligently recognize and locate and grab the target object. The contributions of this paper are: (1) based on SSD algorithm, the network structure is improved according to the demand. The improved recognition rate is higher, the target detection frame is closer to the real part, and the recognition error is greatly reduced. (2) A model of regional posture detection and positioning algorithm is proposed. In the network, the posture angle of the grabbed target is classified and output, and the position coordinates of the grabbed point are output by the regression method. This allows for efficient target recognition and grabbing positioning.


## 2. Related Work

Literature [16] proposed the famous R-CNN target detection model, which combined traditional computer vision technology with deep learning. Because the front convolutional neural network continuously repetitively learned the image redundant features and needed to train the classifier separately, the real-time performance and accuracy were not good. Literature [17] proposed the Fast R-CNN architecture, which first extracted features and then selected candidate target regions. For the first time, end-to-end multitask training had been realized. Literature [18] proposed the Faster R-CNN model. It used a region proposal network, and added anchor boxes and candidate box confidence mechanisms to the network. Literature [19] proposed the YOLOv1 detection model. The network could take the entire picture as the input of the network, and the detection network directly outputted the target category as well as bounding box parameters. Literature [20] proposed an SSD detection model, which proposed feature maps for feature fusion and joint detection. Its performance on small targets was better than YOLOv1, and the anchor frame method was used, which made the detection and positioning accuracy higher. Literature [21] proposed YOLOv2, which removed the classification and positioning of the bounding box. And used the form of anchor box to predict the bounding box, reduced the influence of the loss of spatial feature, improved the positioning accuracy. At the same time, the K-MEANS algorithm was borrowed from bounding box. Literature [22] optimized and proposed YOLOv3 on the basis of V2, network used the new residual network DarkNet-53 instead of DarkNet19.Literature [23] expressed the grabbing parameters as a 2-dimensional oriented rectangle. This kind of robot grabbing the low-dimensional mapping parameter vector had been utilized as a popular method. Some methods utilizing cameras attempted to recover 3D geometry for grabbing [24], where derived feature spaces were used. Deep learning avoided the need for engineering feature space, and needed to rely on larger datasets. First adopted the pretraining method, and then used a relatively small, problem-specific dataset to fine-tune the network model to increase the fit of the network model. According to the sliding window method, literature [25] trained a multimodal network, and the first stage produced more accurate second-stage predictions. Literature [26] first performed the preprocessing step of the image range to identify the candidate object regions, applied the CNN classifier for region. With the development of the target detection architecture and the optimization of the low-dimensional mapping of the grabbing parameters, methods such as the extreme learning machine algorithm [27] or the sparse dictionary learning algorithm [28] were used to detect the grabbing points of the robot’s visual information. Literature [29] used a regression-based method, because the grabbing parameter space was nonconvex. Therefore, it needed to be compensated by image segmentation. Literature [30] conducted experiments on real objects. A two-stage method in literature [31] outputted a learned feature, used it to generate a grabbing output. Literature [32] used a two-stage process, followed by a deep branch. Literature [33] proposed to predict the maximum possible grabbing success parameters through visual input from grab planning. In actual experiments, the process of grabbing from robots was used to learn grabbing experience through reinforcement learning [34]. Literature [35] collected more than 800,000 points, and up to 14 robots run in parallel for learning servo motion.


## 3. Target Recognition and Positioning Based on Improved SSD

The main content of this chapter is based on the improved SSD algorithm to achieve target recognition, classification and positioning. First, the principle of SSD algorithm is introduced. According to the detection requirements of different targets in industrial production, a multiscale feature extraction network is constructed. It integrates the improvement strategy into the SSD network to optimize it, so as to improve the accuracy of target recognition and positioning accuracy.


### 3.1. SSD

Before uploading a color image to the SSD network, it must first be transformed into a fixed-size image. A convolutional neural network (VGG-16) is used to train the SSD system. To anticipate multiscale targets, the fully connected layer in VGG-16 is converted to a convolutional layer and convolutional layers of various sizes are added subsequently. Frame offset and category prediction are handled by multiple tiers of feature maps in the SSD network. Convolutional layers are used to extract features from a picture layer by layer. As a last step, take the feature map and construct a default box for each point. The final test result is produced by the nonmaximum suppression (NMS) operation and the output of the filtered default box. Each receptive field must be classified using a full convolution approach. Therefore, the convolutional layer is used to directly perform regression prediction on target objects of various forms and sizes. To improve detection accuracy and speed, distribute it in multiple feature layers.SSD network divides the sample space into scale-independent layers, with each layer producing a default box. Each of the feature layer’s default candidate boxes predicts target objects at a variety of scales and locations. It is important to note that every pixel on the feature layer serves as the default candidate frame’s center. The SSD network will use a more advanced prediction layer for larger-scale targets in order to extract and forecast target attributes. On the other hand, the detection correlates to the lower layer’s prediction layer for smaller-scale target items.In order to perform regression prediction on target features of different scales, a 3 × 3 convolution kernel is uniformly used in the SSD network. The default candidate boxes of different scales correspond to the corresponding number of 3 × 3 convolution kernels to conduct regression prediction. No regional candidate frames are generated throughout the convolution phase because of SSD’s direct regression prediction method. This dramatically enhances the SSD network’s detecting speed. The structure of the SSD network is simple, the target object prediction method is single, and the detection accuracy is good. As a result of this, the industry’s real-time detection needs can be met, as well as a greater detection rate.


### 3.2. SSD Optimization with Inception Structure

Although the overall performance of the SSD algorithm for target recognition and detection is still considerable. However, when faced with complex situations, sometimes missed detections and false detections may occur. Therefore, this article makes adaptive improvements to the SSD algorithm to improve the network model’s target recognition and detection performance, and thereby improve the effect.SSD networks have a complicated neural network model, and the most common way to improve network performance is to increase network breadth and depth. The problem of overfitting, however, may arise as a result of this additional complexity. Inception has the high performance of a dense matrix, yet it keeps the sparseness of the network structure. GoogleLeNet Inception V1 believes that the primary technique to overcome the problem of higher computation and overfitting caused by parameter increase is to turn the entire connection layer or even half of the convolutional layer into a sparse structure. As a result, the Inception network is able to collect more target data without increasing the complexity of the network. Varying convolution kernels of different sizes are stacked together in the same layer to extract information from the preceding layer, and then stitched together in Inception. This configuration not only expands the network’s width, but it also makes it more flexible in terms of scaling and enables the merging of characteristics from various scales. Image features of varying sizes can be experienced by the same layer network.As the depth increases, the extracted features will become abstract, and each feature involves a larger sensing area. Therefore, a 5 × 5 large-size convolution kernel is added to the Inception structure, but the addition of large convolution kernel causes a increase in calculation. With this in mind, the Inception module uses a 1 × 1 convolution kernel for dimensionality reduction. Choose a 1 × 1 convolution kernel to fuse the feature maps extracted by different convolution kernels. This controls the channel size, increases the nonlinearity of network, and reduces the amount of network calculations.The extra layer of SSD is small in size, contains limited small target information, and the extra layer has only a kind of convolution kernel. The feature maps for additional layers generate target object position and confidence information through convolution operations. In the process of target recognition and detection, the larger-sized convolution kernel is used for learning and prediction of large-scale features, and the small-sized convolution kernel is used for small-scale features. The top feature map of the extra layer may have omissions in the target detailed features. Therefore, this article uses the Inception network structure to replace extra layers in SSD. In order to be able to retain more of the original details of the target, this paper also improves the Inception network structure. Figure1 shows the improved Inception network structure.Figure 1
Improved Inception structure.This article replaces the 5 × 5 kernel with two 3 × 3 kernels. The number of feature maps is decreased, and total number of feature maps is kept the same as the total number of feature maps in the additional layers. To show the importance for different scale convolution kernels of the Inception network structure introduced in this article, this article also uses the batch normalization operation (BN) after each convolution layer in the Inception module. Two conv 1 × 1 convolutional layers are added near the output. In this way, the number of parameters is reduced, the speed is increased. With the deepening and widening of neural networks, the difficulty of feature fusion increases. To solve this issue, residual structure connection is supplemented to reduce the difficulty of feature fusion.This article replaces conv6, conv7 and conv8 with the Inception network structure. Figure2 shows the entire implementation process of the improved SSD algorithm.Figure 2
Improved SSD architecture.In the target detection stage, after the image is input, the image is preprocessed first. Perform grayscale processing on the image first, and then perform Gaussian filter denoising processing to increase the generalization of the image in areas such as brightness, contrast, and different channels. Then randomly expand its size, randomly select the ratio of 0, 0.1, 0.3, 0.7, 0.9 for cropping, and increase the mapping ability of the image part and the whole. Then perform a random mirroring operation, transform from the scaled coordinates to the real coordinates, and finally adjust the image to 300 × 300 pixels through the Resize operation. After the above image preprocessing, it is input to the improved SSD network. First through the VGG16 conv5_3 feature extraction network, output the feature map, after passing through two conv3 × 3 feature extraction networks, output the 10 × 10 scale feature map. Then enter the Inception feature extraction network to obtain a feature map with a scale of 3 × 3. Finally, it passes through a convolutional layer to output a 1 × 1 scale as a feature map. Feature maps of various scales undergo a series of feature detection operations such as convolution, BN, and activation functions to generate and output target category, target location, and unit confidence information. Then generate a default box from the feature map set obtained. Finally, by performing nonmaximum suppression on the detection results of the three scales, the final detection results are output.


### 3.3. Loss Optimization

According to the regression principle of the SSD algorithm in detecting target location and category, whose loss is the combination of confidence loss as well as location loss:(1)LSSD=Lconf+αLlocN,where N is default number of boxes that match the real target box, α is the weight of loss, Lconf is confidence loss, and Lloc is location loss.Aiming at the problem that the SSD model is prone to miss and false detection of stacked targets, the repulsion loss [36] is added on the basis of the original loss function to improve the problem. The improved loss function is as follows:(2)L=LSSD+βLREP,where LREP is repulsion loss and β is the weight.


### 3.4. Nonmaximum Suppression Weight

SSD algorithm is with a feedforward network, the final detection result is obtained through nonmaximum suppression (NMS).B denote the bounding box set, Cj denote the confidence of bounding box, then NMS is(3)BBox=BargmaxCj,where BBox is bounding boxes having the biggest confidence, and bounding boxes will be used as output. Nonmaximum value detection result may contain the maximum value for feature, which is not appropriate to directly call result of nonmaximum value suppression. Supposing all bounding boxes come from the same target, and only considering the maximum result, the problem of utilizing the object information by maintaining the highest confidence of the bounding box will be ignored. In view of the above problems, weighting is performed on the basis of the original nonmaximum suppression, and it is called nonmaximum weight (NMW):(4)BBox=∑j=1nwjBj∑j=1nwj,wj=Cj×IOUBj,BargmaxCj,where n is the number of bounding boxes and w is the weight of bounding box.


## 3.1. SSD

Before uploading a color image to the SSD network, it must first be transformed into a fixed-size image. A convolutional neural network (VGG-16) is used to train the SSD system. To anticipate multiscale targets, the fully connected layer in VGG-16 is converted to a convolutional layer and convolutional layers of various sizes are added subsequently. Frame offset and category prediction are handled by multiple tiers of feature maps in the SSD network. Convolutional layers are used to extract features from a picture layer by layer. As a last step, take the feature map and construct a default box for each point. The final test result is produced by the nonmaximum suppression (NMS) operation and the output of the filtered default box. Each receptive field must be classified using a full convolution approach. Therefore, the convolutional layer is used to directly perform regression prediction on target objects of various forms and sizes. To improve detection accuracy and speed, distribute it in multiple feature layers.SSD network divides the sample space into scale-independent layers, with each layer producing a default box. Each of the feature layer’s default candidate boxes predicts target objects at a variety of scales and locations. It is important to note that every pixel on the feature layer serves as the default candidate frame’s center. The SSD network will use a more advanced prediction layer for larger-scale targets in order to extract and forecast target attributes. On the other hand, the detection correlates to the lower layer’s prediction layer for smaller-scale target items.In order to perform regression prediction on target features of different scales, a 3 × 3 convolution kernel is uniformly used in the SSD network. The default candidate boxes of different scales correspond to the corresponding number of 3 × 3 convolution kernels to conduct regression prediction. No regional candidate frames are generated throughout the convolution phase because of SSD’s direct regression prediction method. This dramatically enhances the SSD network’s detecting speed. The structure of the SSD network is simple, the target object prediction method is single, and the detection accuracy is good. As a result of this, the industry’s real-time detection needs can be met, as well as a greater detection rate.


## 3.2. SSD Optimization with Inception Structure

Although the overall performance of the SSD algorithm for target recognition and detection is still considerable. However, when faced with complex situations, sometimes missed detections and false detections may occur. Therefore, this article makes adaptive improvements to the SSD algorithm to improve the network model’s target recognition and detection performance, and thereby improve the effect.SSD networks have a complicated neural network model, and the most common way to improve network performance is to increase network breadth and depth. The problem of overfitting, however, may arise as a result of this additional complexity. Inception has the high performance of a dense matrix, yet it keeps the sparseness of the network structure. GoogleLeNet Inception V1 believes that the primary technique to overcome the problem of higher computation and overfitting caused by parameter increase is to turn the entire connection layer or even half of the convolutional layer into a sparse structure. As a result, the Inception network is able to collect more target data without increasing the complexity of the network. Varying convolution kernels of different sizes are stacked together in the same layer to extract information from the preceding layer, and then stitched together in Inception. This configuration not only expands the network’s width, but it also makes it more flexible in terms of scaling and enables the merging of characteristics from various scales. Image features of varying sizes can be experienced by the same layer network.As the depth increases, the extracted features will become abstract, and each feature involves a larger sensing area. Therefore, a 5 × 5 large-size convolution kernel is added to the Inception structure, but the addition of large convolution kernel causes a increase in calculation. With this in mind, the Inception module uses a 1 × 1 convolution kernel for dimensionality reduction. Choose a 1 × 1 convolution kernel to fuse the feature maps extracted by different convolution kernels. This controls the channel size, increases the nonlinearity of network, and reduces the amount of network calculations.The extra layer of SSD is small in size, contains limited small target information, and the extra layer has only a kind of convolution kernel. The feature maps for additional layers generate target object position and confidence information through convolution operations. In the process of target recognition and detection, the larger-sized convolution kernel is used for learning and prediction of large-scale features, and the small-sized convolution kernel is used for small-scale features. The top feature map of the extra layer may have omissions in the target detailed features. Therefore, this article uses the Inception network structure to replace extra layers in SSD. In order to be able to retain more of the original details of the target, this paper also improves the Inception network structure. Figure1 shows the improved Inception network structure.Figure 1
Improved Inception structure.This article replaces the 5 × 5 kernel with two 3 × 3 kernels. The number of feature maps is decreased, and total number of feature maps is kept the same as the total number of feature maps in the additional layers. To show the importance for different scale convolution kernels of the Inception network structure introduced in this article, this article also uses the batch normalization operation (BN) after each convolution layer in the Inception module. Two conv 1 × 1 convolutional layers are added near the output. In this way, the number of parameters is reduced, the speed is increased. With the deepening and widening of neural networks, the difficulty of feature fusion increases. To solve this issue, residual structure connection is supplemented to reduce the difficulty of feature fusion.This article replaces conv6, conv7 and conv8 with the Inception network structure. Figure2 shows the entire implementation process of the improved SSD algorithm.Figure 2
Improved SSD architecture.In the target detection stage, after the image is input, the image is preprocessed first. Perform grayscale processing on the image first, and then perform Gaussian filter denoising processing to increase the generalization of the image in areas such as brightness, contrast, and different channels. Then randomly expand its size, randomly select the ratio of 0, 0.1, 0.3, 0.7, 0.9 for cropping, and increase the mapping ability of the image part and the whole. Then perform a random mirroring operation, transform from the scaled coordinates to the real coordinates, and finally adjust the image to 300 × 300 pixels through the Resize operation. After the above image preprocessing, it is input to the improved SSD network. First through the VGG16 conv5_3 feature extraction network, output the feature map, after passing through two conv3 × 3 feature extraction networks, output the 10 × 10 scale feature map. Then enter the Inception feature extraction network to obtain a feature map with a scale of 3 × 3. Finally, it passes through a convolutional layer to output a 1 × 1 scale as a feature map. Feature maps of various scales undergo a series of feature detection operations such as convolution, BN, and activation functions to generate and output target category, target location, and unit confidence information. Then generate a default box from the feature map set obtained. Finally, by performing nonmaximum suppression on the detection results of the three scales, the final detection results are output.


## 3.3. Loss Optimization

According to the regression principle of the SSD algorithm in detecting target location and category, whose loss is the combination of confidence loss as well as location loss:(1)LSSD=Lconf+αLlocN,where N is default number of boxes that match the real target box, α is the weight of loss, Lconf is confidence loss, and Lloc is location loss.Aiming at the problem that the SSD model is prone to miss and false detection of stacked targets, the repulsion loss [36] is added on the basis of the original loss function to improve the problem. The improved loss function is as follows:(2)L=LSSD+βLREP,where LREP is repulsion loss and β is the weight.


## 3.4. Nonmaximum Suppression Weight

SSD algorithm is with a feedforward network, the final detection result is obtained through nonmaximum suppression (NMS).B denote the bounding box set, Cj denote the confidence of bounding box, then NMS is(3)BBox=BargmaxCj,where BBox is bounding boxes having the biggest confidence, and bounding boxes will be used as output. Nonmaximum value detection result may contain the maximum value for feature, which is not appropriate to directly call result of nonmaximum value suppression. Supposing all bounding boxes come from the same target, and only considering the maximum result, the problem of utilizing the object information by maintaining the highest confidence of the bounding box will be ignored. In view of the above problems, weighting is performed on the basis of the original nonmaximum suppression, and it is called nonmaximum weight (NMW):(4)BBox=∑j=1nwjBj∑j=1nwj,wj=Cj×IOUBj,BargmaxCj,where n is the number of bounding boxes and w is the weight of bounding box.


## 4. Region Posture Detection and Grabbing Positioning

After using the improved SSD in the previous section to identify and detect the target, the output of the previous step needs to be placed in the input network of the next stage to perform posture detection and grabbed point positioning of the grabbed target.


### 4.1. Mapping Expression of the Target in the Image Space

The formula of 5-dimensional grabbing parameters is as follows:(5)G=r,c,m,n,θ,where r,c represents the coordinates of the upper left corner for grabbing matrix, the gripping width is m, the gripping opening size is n, and the angle between it and the X axis is θ. Such 5-dimensional grabbing matrix parameters can be suitable for deep learning for grabbing parameter prediction. The 5-dimensional grabbing parameters can not only be used for direct regression convolutional neural networks, but also match the input and output of the target detection network.(5) cannot accurately represent the grab point and grab frame parameter information, but it can represent the grab angle of the grab point. After optimization, the current five-dimensional grabbing parameters are as follows. The optimized grabbing parameters can express the grabbing task in the form of grabbing frames, which can reflect the position of the grabbing point required for grabbing and the rotation angle required for grabbing.(6)G=x,y,w,h,θ,where x,y represents the best grab point coordinates of the grab target. The width of w, that is, the width of the parallel splint that needs to be opened during grabbing, and the height of h are physical constraints. θ is the attitude angle of the rectangle relative to the horizontal axis, that is, the angle that the machine splint needs to rotate.A grab target can have many suitable grab points, and the direction of each grab point is different from the opening size of the fixture. The deep learning target recognition detection model is applied to grabbing and positioning, and the previous grabbing regression task is turned into a detection task. For the same grabbing target picture, multiple grabbing matrix parameters that can be grabbed can be output, and the optimal grabbing point position and corresponding posture angle can also be selected.


### 4.2. Network Structure

Figure3 is the network structure of multigrabbing target area posture detection and algorithm, and the grabbing parameters are expressed as grabbing matrices. The regional pose detection and positioning model uses regression to output the coordinates and constraint values of the grab points, and outputs the rotation angle of the grab matrix in the form of category labels.Figure 3
Regional pose detection and grabbing positioning model.The detection network does not use the Selective search algorithm to randomly select an area in the image as a possible capture frame position input to the network, avoiding repeated learning of redundant background information. A grabbing target in an image may have multiple grabbing rectangles corresponding to the grabbing positions, and the model can predict multiple grabbing boxes for each grabbing target and the corresponding confidence scores. In the regional pose detection and positioning algorithm model, the InceptionNet network is used as the basic network of the entire model. The regional posture detection and grabbing and positioning algorithm adopts a distributed loss function calculation method, so that each sub-network has a different learning ability. This improves the feature extraction and feature learning capabilities of the entire algorithm model. The InceptionV2 network is used in the regional attitude detection and positioning network, and batch normalization (BN) is added to network. This reduces the influence of singular values on the distribution of neuron data. While increasing the depth and width of the regional pose detection and algorithm network, a 1 × 1 convolutional layer is used to replace fully connected layer, which greatly reduces the number of parameters in the grab point regression and pose classification. In the Inception module, multichannel convolution is performed in the form of convolution kernel splicing and parallel connection, which greatly reduces the number of weights of the convolution kernel and the fully connected layer. This speeds up the calculation speed of the model, and increases the generalization ability and nonlinear expansion expression ability of the regional attitude detection and positioning model.


#### 4.2.1. Posture Angle as Classification Task

In this research, the 5-dimensional grabbing parameters are output by the detection method, and an algorithm for the detection and positioning of the area grabbing target posture is proposed. In this research, the position coordinates of the grabbing frame and the angle of the grabbing posture are predicted in parallel, and the regression methodx,y,w,h,θ is used to predict the position of the grabbing point in the algorithm model. The rotatable grabbing angle of the machine arm is quantified as K angle intervals, and the grabbing angle corresponds to the label of a certain grabbing interval. Therefore, the capture angle is input and output in the algorithm model in the form of classification labels. Since the algorithm will have no capture target in the image during the prediction process, it is necessary to add a capture target category to indicate that there is no tag output of the capture target in the image. In addition, the total number of categories is K+1 categories. The range of mechanical rotation angle is −90,90. The rotation angle is uniformly quantized into 18 subintervals, and each subinterval is incremented by 10 increments. Each subinterval corresponds to a classification label li, and the classification label has a unique angle value θi corresponding to it. This paper uses this quantitative method to transform the posture angle regression task into a classification task.


#### 4.2.2. Grabbing Region Proposal

A grabbing region proposal network is embedded to entire pipeline to predict all possible approximate capture locations for the entire image of the input model. Since all the grabbing frames detected in the image are rectangular boxes parallel to theX axis and Y-axis, and the marked grabbing area is a detection frame with an angle θ. Align the grabbing frame with the detection form, and rotate the grabbing frame with angle θ counterclockwise. Rotate the grabbing frame to be parallel to the X axis and Y axis. Learning features for grabbing direction, edge features, and line segment in the grabbing frame is used to predict the corresponding angle category.In order to simplify the difficulty and calculation amount of the network in feature learning, avoid redundant learning of the features of an image. Before the candidate capture area network, the shared convolutional layer is utilized to learn features for input image. After shared convolutional layer, the convolutional features are respectively transmitted to the candidate capture area and the perceptual area pooling layer. The grabbed region proposal is used to predict the location and type of the capture frame in the output feature map and the corresponding confidence score. Pooling is performed on the mapping of the grabbing region of the original image on the feature map after convolution. Perceptual area pooling fixes the scale of the input feature map and merges it into a new dimension feature map.Figure4 is a grabbing region proposal network, and the feature map after shared convolution is input to the grabbing region proposal network.Figure 4
Grabbing region proposal.Use a 3 × 3 convolution kernel on the feature map to convolve and output a feature map of 96 channels. It uses ZFnet as the basic network to predict that the feature map contains the target area and does not contain the target area parameters. After 5 layers of convolution, finally output a 512-dimensional feature map of the grabbed candidate area. These features are connected using a fully connected layer to output the capture point position and attitude angle category to predict the possible capture frame and the corresponding confidence score. Each possible grab box has two confidence levels corresponding to it, which means that the grab box contains the probability of the grab point, and the two probabilities are opposite and unified.The loss of grabbing region proposal network is:(7)LGR=1Ngr_cls∑iLgr_clspi,pi∗+γ1Ngr_reg∑ipi∗Lgr_regti,ti∗,where ti represents the 4-dimensional vector of the parameter x,y,w,h of the i-th prediction capture matrix. pi represents the probability that the predicted grab angle label is the i-th label. pi=0 represents the part of the background of the image. pi∗=1 indicates that there are candidate grabbing frames and their corresponding angle labels. ti∗ is the coordinate vector of actual grab box corresponding pi∗. Where Lgr_cls and Lgr_reg respectively represent the posture angle label classification loss function and the grab box position regression loss function. Ngr_cls and Ngr_reg represent normalization parameters. γ represents the normalized balance weight.


#### 4.2.3. Grabbing Target Posture Detection and Positioning

In the area grabbing target pose detection and positioning algorithm network, the candidate grabbing area network and the interesting area pooling are inserted into the IncepionV2 network. The multisegment shared convolutional network method is adopted to avoid redundant feature extraction and learning of repeated feature maps. After the entire InceptionV2 is connected to two parallel fully connected networks, the Softmax function is used to calculate the probability of the output attitude angle belonging to each category. The L1 regression loss function is used to calculate the Euclidean distance from the actual grab frame coordinates when outputting the grab frame parameter coordinates. Continuously iteratively update the network parameters to output more accurate nondirectionalx,y,w,h grab frame coordinates and corresponding attitude angle labels. In the network, nonpositioned grabbing frames are used for learning and prediction, and a precise and refined grabbing frame is made through the output of candidate grabbing regions. Define the end-to-end capture frame parameter regression and capture target pose angle classification label loss function as:(8)LPC=1Npc_cls∑iLpc_clsqi,qi∗+μ1Npc_reg∑iqi∗Lpc_regvi,vi∗,where qi represents the probability of predicting that the capture angle label is the i-th label after the Softmax function. vi represents the 4-dimensional vector of the i-th grabbing matrix parameter x,y,w,h predicted by the network. Where qi∗=0 represents the background part of the image. qi∗=1 indicates that there are candidate grabbing frames and their corresponding angle labels. The variable vi∗ represents the 4-dimensional capture coordinate vector of the actual capture frame corresponding to the qi∗ label of the actual target area. Lpc_cls and Lpc_reg respectively represent the posture angle label classification loss function and the grab box position regression loss function. Npc_cls and Npc_reg represent the influence on loss and average value to reduce the value. μ is the total loss for regression loss and balance weight. Therefore, the total loss function of the network end-to-end training is:(9)Ltotal=LGR+LPC.In the network, a batch of input feature data will be normalized before the activation function of the convolutional layer of the inception module branch. Let the front convolutional layer output a batch of eigenvalues after batch normalization, and use mathematical statistics to make the input data be normally distributed, and each eigenvalue falls in the distribution domain ofN0,1. The input eigenvalues will not fall in the saturation region of the activation function, and the influence of singular values on the distribution of neuron data will be reduced.


## 4.1. Mapping Expression of the Target in the Image Space

The formula of 5-dimensional grabbing parameters is as follows:(5)G=r,c,m,n,θ,where r,c represents the coordinates of the upper left corner for grabbing matrix, the gripping width is m, the gripping opening size is n, and the angle between it and the X axis is θ. Such 5-dimensional grabbing matrix parameters can be suitable for deep learning for grabbing parameter prediction. The 5-dimensional grabbing parameters can not only be used for direct regression convolutional neural networks, but also match the input and output of the target detection network.(5) cannot accurately represent the grab point and grab frame parameter information, but it can represent the grab angle of the grab point. After optimization, the current five-dimensional grabbing parameters are as follows. The optimized grabbing parameters can express the grabbing task in the form of grabbing frames, which can reflect the position of the grabbing point required for grabbing and the rotation angle required for grabbing.(6)G=x,y,w,h,θ,where x,y represents the best grab point coordinates of the grab target. The width of w, that is, the width of the parallel splint that needs to be opened during grabbing, and the height of h are physical constraints. θ is the attitude angle of the rectangle relative to the horizontal axis, that is, the angle that the machine splint needs to rotate.A grab target can have many suitable grab points, and the direction of each grab point is different from the opening size of the fixture. The deep learning target recognition detection model is applied to grabbing and positioning, and the previous grabbing regression task is turned into a detection task. For the same grabbing target picture, multiple grabbing matrix parameters that can be grabbed can be output, and the optimal grabbing point position and corresponding posture angle can also be selected.


## 4.2. Network Structure

Figure3 is the network structure of multigrabbing target area posture detection and algorithm, and the grabbing parameters are expressed as grabbing matrices. The regional pose detection and positioning model uses regression to output the coordinates and constraint values of the grab points, and outputs the rotation angle of the grab matrix in the form of category labels.Figure 3
Regional pose detection and grabbing positioning model.The detection network does not use the Selective search algorithm to randomly select an area in the image as a possible capture frame position input to the network, avoiding repeated learning of redundant background information. A grabbing target in an image may have multiple grabbing rectangles corresponding to the grabbing positions, and the model can predict multiple grabbing boxes for each grabbing target and the corresponding confidence scores. In the regional pose detection and positioning algorithm model, the InceptionNet network is used as the basic network of the entire model. The regional posture detection and grabbing and positioning algorithm adopts a distributed loss function calculation method, so that each sub-network has a different learning ability. This improves the feature extraction and feature learning capabilities of the entire algorithm model. The InceptionV2 network is used in the regional attitude detection and positioning network, and batch normalization (BN) is added to network. This reduces the influence of singular values on the distribution of neuron data. While increasing the depth and width of the regional pose detection and algorithm network, a 1 × 1 convolutional layer is used to replace fully connected layer, which greatly reduces the number of parameters in the grab point regression and pose classification. In the Inception module, multichannel convolution is performed in the form of convolution kernel splicing and parallel connection, which greatly reduces the number of weights of the convolution kernel and the fully connected layer. This speeds up the calculation speed of the model, and increases the generalization ability and nonlinear expansion expression ability of the regional attitude detection and positioning model.


### 4.2.1. Posture Angle as Classification Task

In this research, the 5-dimensional grabbing parameters are output by the detection method, and an algorithm for the detection and positioning of the area grabbing target posture is proposed. In this research, the position coordinates of the grabbing frame and the angle of the grabbing posture are predicted in parallel, and the regression methodx,y,w,h,θ is used to predict the position of the grabbing point in the algorithm model. The rotatable grabbing angle of the machine arm is quantified as K angle intervals, and the grabbing angle corresponds to the label of a certain grabbing interval. Therefore, the capture angle is input and output in the algorithm model in the form of classification labels. Since the algorithm will have no capture target in the image during the prediction process, it is necessary to add a capture target category to indicate that there is no tag output of the capture target in the image. In addition, the total number of categories is K+1 categories. The range of mechanical rotation angle is −90,90. The rotation angle is uniformly quantized into 18 subintervals, and each subinterval is incremented by 10 increments. Each subinterval corresponds to a classification label li, and the classification label has a unique angle value θi corresponding to it. This paper uses this quantitative method to transform the posture angle regression task into a classification task.


### 4.2.2. Grabbing Region Proposal

A grabbing region proposal network is embedded to entire pipeline to predict all possible approximate capture locations for the entire image of the input model. Since all the grabbing frames detected in the image are rectangular boxes parallel to theX axis and Y-axis, and the marked grabbing area is a detection frame with an angle θ. Align the grabbing frame with the detection form, and rotate the grabbing frame with angle θ counterclockwise. Rotate the grabbing frame to be parallel to the X axis and Y axis. Learning features for grabbing direction, edge features, and line segment in the grabbing frame is used to predict the corresponding angle category.In order to simplify the difficulty and calculation amount of the network in feature learning, avoid redundant learning of the features of an image. Before the candidate capture area network, the shared convolutional layer is utilized to learn features for input image. After shared convolutional layer, the convolutional features are respectively transmitted to the candidate capture area and the perceptual area pooling layer. The grabbed region proposal is used to predict the location and type of the capture frame in the output feature map and the corresponding confidence score. Pooling is performed on the mapping of the grabbing region of the original image on the feature map after convolution. Perceptual area pooling fixes the scale of the input feature map and merges it into a new dimension feature map.Figure4 is a grabbing region proposal network, and the feature map after shared convolution is input to the grabbing region proposal network.Figure 4
Grabbing region proposal.Use a 3 × 3 convolution kernel on the feature map to convolve and output a feature map of 96 channels. It uses ZFnet as the basic network to predict that the feature map contains the target area and does not contain the target area parameters. After 5 layers of convolution, finally output a 512-dimensional feature map of the grabbed candidate area. These features are connected using a fully connected layer to output the capture point position and attitude angle category to predict the possible capture frame and the corresponding confidence score. Each possible grab box has two confidence levels corresponding to it, which means that the grab box contains the probability of the grab point, and the two probabilities are opposite and unified.The loss of grabbing region proposal network is:(7)LGR=1Ngr_cls∑iLgr_clspi,pi∗+γ1Ngr_reg∑ipi∗Lgr_regti,ti∗,where ti represents the 4-dimensional vector of the parameter x,y,w,h of the i-th prediction capture matrix. pi represents the probability that the predicted grab angle label is the i-th label. pi=0 represents the part of the background of the image. pi∗=1 indicates that there are candidate grabbing frames and their corresponding angle labels. ti∗ is the coordinate vector of actual grab box corresponding pi∗. Where Lgr_cls and Lgr_reg respectively represent the posture angle label classification loss function and the grab box position regression loss function. Ngr_cls and Ngr_reg represent normalization parameters. γ represents the normalized balance weight.


### 4.2.3. Grabbing Target Posture Detection and Positioning

In the area grabbing target pose detection and positioning algorithm network, the candidate grabbing area network and the interesting area pooling are inserted into the IncepionV2 network. The multisegment shared convolutional network method is adopted to avoid redundant feature extraction and learning of repeated feature maps. After the entire InceptionV2 is connected to two parallel fully connected networks, the Softmax function is used to calculate the probability of the output attitude angle belonging to each category. The L1 regression loss function is used to calculate the Euclidean distance from the actual grab frame coordinates when outputting the grab frame parameter coordinates. Continuously iteratively update the network parameters to output more accurate nondirectionalx,y,w,h grab frame coordinates and corresponding attitude angle labels. In the network, nonpositioned grabbing frames are used for learning and prediction, and a precise and refined grabbing frame is made through the output of candidate grabbing regions. Define the end-to-end capture frame parameter regression and capture target pose angle classification label loss function as:(8)LPC=1Npc_cls∑iLpc_clsqi,qi∗+μ1Npc_reg∑iqi∗Lpc_regvi,vi∗,where qi represents the probability of predicting that the capture angle label is the i-th label after the Softmax function. vi represents the 4-dimensional vector of the i-th grabbing matrix parameter x,y,w,h predicted by the network. Where qi∗=0 represents the background part of the image. qi∗=1 indicates that there are candidate grabbing frames and their corresponding angle labels. The variable vi∗ represents the 4-dimensional capture coordinate vector of the actual capture frame corresponding to the qi∗ label of the actual target area. Lpc_cls and Lpc_reg respectively represent the posture angle label classification loss function and the grab box position regression loss function. Npc_cls and Npc_reg represent the influence on loss and average value to reduce the value. μ is the total loss for regression loss and balance weight. Therefore, the total loss function of the network end-to-end training is:(9)Ltotal=LGR+LPC.In the network, a batch of input feature data will be normalized before the activation function of the convolutional layer of the inception module branch. Let the front convolutional layer output a batch of eigenvalues after batch normalization, and use mathematical statistics to make the input data be normally distributed, and each eigenvalue falls in the distribution domain ofN0,1. The input eigenvalues will not fall in the saturation region of the activation function, and the influence of singular values on the distribution of neuron data will be reduced.


## 4.2.1. Posture Angle as Classification Task

In this research, the 5-dimensional grabbing parameters are output by the detection method, and an algorithm for the detection and positioning of the area grabbing target posture is proposed. In this research, the position coordinates of the grabbing frame and the angle of the grabbing posture are predicted in parallel, and the regression methodx,y,w,h,θ is used to predict the position of the grabbing point in the algorithm model. The rotatable grabbing angle of the machine arm is quantified as K angle intervals, and the grabbing angle corresponds to the label of a certain grabbing interval. Therefore, the capture angle is input and output in the algorithm model in the form of classification labels. Since the algorithm will have no capture target in the image during the prediction process, it is necessary to add a capture target category to indicate that there is no tag output of the capture target in the image. In addition, the total number of categories is K+1 categories. The range of mechanical rotation angle is −90,90. The rotation angle is uniformly quantized into 18 subintervals, and each subinterval is incremented by 10 increments. Each subinterval corresponds to a classification label li, and the classification label has a unique angle value θi corresponding to it. This paper uses this quantitative method to transform the posture angle regression task into a classification task.


## 4.2.2. Grabbing Region Proposal

A grabbing region proposal network is embedded to entire pipeline to predict all possible approximate capture locations for the entire image of the input model. Since all the grabbing frames detected in the image are rectangular boxes parallel to theX axis and Y-axis, and the marked grabbing area is a detection frame with an angle θ. Align the grabbing frame with the detection form, and rotate the grabbing frame with angle θ counterclockwise. Rotate the grabbing frame to be parallel to the X axis and Y axis. Learning features for grabbing direction, edge features, and line segment in the grabbing frame is used to predict the corresponding angle category.In order to simplify the difficulty and calculation amount of the network in feature learning, avoid redundant learning of the features of an image. Before the candidate capture area network, the shared convolutional layer is utilized to learn features for input image. After shared convolutional layer, the convolutional features are respectively transmitted to the candidate capture area and the perceptual area pooling layer. The grabbed region proposal is used to predict the location and type of the capture frame in the output feature map and the corresponding confidence score. Pooling is performed on the mapping of the grabbing region of the original image on the feature map after convolution. Perceptual area pooling fixes the scale of the input feature map and merges it into a new dimension feature map.Figure4 is a grabbing region proposal network, and the feature map after shared convolution is input to the grabbing region proposal network.Figure 4
Grabbing region proposal.Use a 3 × 3 convolution kernel on the feature map to convolve and output a feature map of 96 channels. It uses ZFnet as the basic network to predict that the feature map contains the target area and does not contain the target area parameters. After 5 layers of convolution, finally output a 512-dimensional feature map of the grabbed candidate area. These features are connected using a fully connected layer to output the capture point position and attitude angle category to predict the possible capture frame and the corresponding confidence score. Each possible grab box has two confidence levels corresponding to it, which means that the grab box contains the probability of the grab point, and the two probabilities are opposite and unified.The loss of grabbing region proposal network is:(7)LGR=1Ngr_cls∑iLgr_clspi,pi∗+γ1Ngr_reg∑ipi∗Lgr_regti,ti∗,where ti represents the 4-dimensional vector of the parameter x,y,w,h of the i-th prediction capture matrix. pi represents the probability that the predicted grab angle label is the i-th label. pi=0 represents the part of the background of the image. pi∗=1 indicates that there are candidate grabbing frames and their corresponding angle labels. ti∗ is the coordinate vector of actual grab box corresponding pi∗. Where Lgr_cls and Lgr_reg respectively represent the posture angle label classification loss function and the grab box position regression loss function. Ngr_cls and Ngr_reg represent normalization parameters. γ represents the normalized balance weight.


## 4.2.3. Grabbing Target Posture Detection and Positioning

In the area grabbing target pose detection and positioning algorithm network, the candidate grabbing area network and the interesting area pooling are inserted into the IncepionV2 network. The multisegment shared convolutional network method is adopted to avoid redundant feature extraction and learning of repeated feature maps. After the entire InceptionV2 is connected to two parallel fully connected networks, the Softmax function is used to calculate the probability of the output attitude angle belonging to each category. The L1 regression loss function is used to calculate the Euclidean distance from the actual grab frame coordinates when outputting the grab frame parameter coordinates. Continuously iteratively update the network parameters to output more accurate nondirectionalx,y,w,h grab frame coordinates and corresponding attitude angle labels. In the network, nonpositioned grabbing frames are used for learning and prediction, and a precise and refined grabbing frame is made through the output of candidate grabbing regions. Define the end-to-end capture frame parameter regression and capture target pose angle classification label loss function as:(8)LPC=1Npc_cls∑iLpc_clsqi,qi∗+μ1Npc_reg∑iqi∗Lpc_regvi,vi∗,where qi represents the probability of predicting that the capture angle label is the i-th label after the Softmax function. vi represents the 4-dimensional vector of the i-th grabbing matrix parameter x,y,w,h predicted by the network. Where qi∗=0 represents the background part of the image. qi∗=1 indicates that there are candidate grabbing frames and their corresponding angle labels. The variable vi∗ represents the 4-dimensional capture coordinate vector of the actual capture frame corresponding to the qi∗ label of the actual target area. Lpc_cls and Lpc_reg respectively represent the posture angle label classification loss function and the grab box position regression loss function. Npc_cls and Npc_reg represent the influence on loss and average value to reduce the value. μ is the total loss for regression loss and balance weight. Therefore, the total loss function of the network end-to-end training is:(9)Ltotal=LGR+LPC.In the network, a batch of input feature data will be normalized before the activation function of the convolutional layer of the inception module branch. Let the front convolutional layer output a batch of eigenvalues after batch normalization, and use mathematical statistics to make the input data be normally distributed, and each eigenvalue falls in the distribution domain ofN0,1. The input eigenvalues will not fall in the saturation region of the activation function, and the influence of singular values on the distribution of neuron data will be reduced.


## 5. Experiment and Discussion

### 5.1. Dataset and Evaluation Metric

This article uses two self-made datasets to experiment on the network: industrial parts (IP) dataset and daily necessities (DN) dataset. IP dataset contains 7852 images, of which 6281 images are training set, and the remaining 1571 images are testing set. DS dataset contains 9835 images, of which 7866 images are training set, and the remaining 1969 images are testing set.When performing target recognition and detection, the evaluation indicators used are precision, recall, and mAP. When grabbing and positioning, the judgment standard consists of two parts. The first part predicts the closeness of the grabbing frame to the actual marked grabbing frame, and the similarity in position and area. The second part is whether the predicted rotation angle label of the grab point and the actually marked grab angle label are within the acceptable range. If both of them meet the judgment criteria, it can be considered that the prediction is accurate, and it is a good prediction of the grabbing point and the grabbing rotation angle. The conditions that need to be met are(10)gtθ−gpθ≤30,gp∩gtgp∪gt>0.25,where gp represents the grab point grab box x,y,w,h parameters predicted by the model. gt represents the actual grabbing frame parameters of the image annotation. gpθ indicates that the model predicts the corresponding angle value of the grab rotation angle label of the grab point. gtθ represents the angle value corresponding to the capture rotation angle label of the actual marked capture point in the image.


### 5.2. Evaluation on Target Recognition

In this work, an improved SSD (ISSD) is proposed to perform the target recognition detection and positioning. To verify the effectiveness of our designed method, we first compare this method with other methods. The result is illustrated in Table1.Table 1
Comparison of ISSD and other methods.
MethodIPDNPrecisionRecallmAPPrecisionRecallmAPFaster-RCNN85.481.773.584.380.972.6YOLO89.285.377.887.984.775.9SSD91.387.480.690.486.579.2ISSD (ours)93.489.284.992.188.183.6It can be seen that, compared with other identification detection methods, the method used in this article can obtain the best performance. The precision of 93.4% and 92.1%, recall of 89.2% and 88.1%, mAP of 84.9% and 83.6% can be obtained on the two datasets. Compared with traditional SSD, the improved algorithm can obtain 2.1% and 1.7% gains on precision, 1.8% and 1.6% gains on recall, 4.3% and 4.4% gains on mAP. These data can verify the correctness of our method.As mentioned early, the improved inception structure is applied to optimize SSD. To verify the effectiveness of this improvement strategy, this article compares the recognition and detection performance of using improved Inception structure and not using this structure. The result is illustrated in Figure5.Figure 5
Evaluation on improved Inception.Obviously, when the Inception structure is not used, the performance of the network is the lowest. After using this structure, the performance of the model will be improved to a certain extent. Especially after improving the Inception model, the performance will be further improved. This shows the correctness of the combination of the Inception structure and the SSD model in this article, and also proves the effectiveness of the improved strategy proposed in this article for the Inception structure.Besides, in the used loss of SSD, this work combines the traditional loss with repulsion loss. To verify the effectiveness of this optimization strategy for network loss, this article compares the recognition and detection performance of using the combined loss and using the single traditional loss. The result is illustrated in Figure6.Figure 6
Evaluation on improved loss.Obviously, when the repulsion loss is not used, the performance of the network is lower. After using the combined loss, the performance of the model will be improved to a certain extent. This shows the correctness of the combination of the traditional SSD loss with repulsion loss in this article, and also proves the effectiveness of the improved strategy proposed in this article for the loss. This loss can guide the network to extract more efficient feature.Last, when performing target recognition and detection, this paper improves the nonmaximum suppression algorithm, and proposes a nonmaximum suppression weight algorithm (NMSW). To verify the effectiveness of this improvement strategy, this work compares the recognition and detection performance of using NMSM and using the traditional NMS. The result is illustrated in Figure7.Figure 7
Evaluation on NMSW.It can be seen that, compared with the traditional NMS method, the NMSW method proposed in this paper can achieve higher performance. On the two datasets, the mAP of 2.3% and 2.1% are improved, respectively. This verifies the correctness of the proposed NMSW algorithm. This is mainly because NMSW overcomes the shortcomings of traditional NMS methods that are easy to ignore by keeping the highest confidence of the bounding box to make full use of the object information.


### 5.3. Evaluation on Grabbing Positioning

In this paper, the trained grabbing positioning model (GPM) is tested, and the performance of the algorithm model is tested. The test set image is composed of the target data set output by the improved SSD target recognition and detection model in the first stage. The detection accuracy is composed of two parts, one is instance detection (ID). That is, the grabbing targets included in the test set are all grabbing targets that appear in the training set and appear at different positions and angles. This detection performance can be suitable for industrial machines to perform posture detection and positioning of grabbed targets of known target categories. The second is object detection (OD), that is, the capture target categories that appear in the test set are not in the network model training set. This detection performance can judge the versatility and adaptability of a machine. High-precision regional posture detection and grabbing point positioning can also be performed for grabbing targets with new shapes. The comparison result with other methods is illustrated in Table2.Table 2
Comparison of GPM and other methods.
MethodIPDNIDODIDODRTGD88.487.988.187.4HCF90.289.589.688.7SDL92.792.291.590.8MOMG95.394.693.993.2GPM (ours)96.596.295.795.1It can be seen that, compared with other grabbing positioning methods, the method used in this work can obtain the best performance. The instance detection accuracy of 96.5% and 95.7%, object detection accuracy of 96.2% and 95.1% can be obtained on the two datasets. Compared with the best listed method, the proposed algorithm can obtain 1.2% and 1.8% gains on instance detection accuracy, 1.6% and 1.9% gains on object detection accuracy. These data can verify the correctness of our method.As mentioned early, this work will normalize a batch of input feature data before the activation function of the convolutional layer of the inception module branch. Using this method, the input data is normally distributed, and each eigenvalue falls in the standard distribution domain. The input eigenvalues will not fall in the saturation region of the activation function, and the influence of singular values on the distribution of neuron data will be reduced. In order to verify the effectiveness of this strategy, this paper compares the network performance when BN is not used with the network performance when BN is used. The result is shown in Figure8.Figure 8
Evaluation on BN layer.Obviously, with the introduction of the BN layer, the performance of the network has been improved. On the IP dataset, the improvement of the two performance indicators is 2.3% and 2.4% respectively. On the DN dataset, the improvement of the two performance indicators is 1.4% and 1.5% respectively. These data verify the effectiveness and correctness of our proposed strategy.


## 5.1. Dataset and Evaluation Metric

This article uses two self-made datasets to experiment on the network: industrial parts (IP) dataset and daily necessities (DN) dataset. IP dataset contains 7852 images, of which 6281 images are training set, and the remaining 1571 images are testing set. DS dataset contains 9835 images, of which 7866 images are training set, and the remaining 1969 images are testing set.When performing target recognition and detection, the evaluation indicators used are precision, recall, and mAP. When grabbing and positioning, the judgment standard consists of two parts. The first part predicts the closeness of the grabbing frame to the actual marked grabbing frame, and the similarity in position and area. The second part is whether the predicted rotation angle label of the grab point and the actually marked grab angle label are within the acceptable range. If both of them meet the judgment criteria, it can be considered that the prediction is accurate, and it is a good prediction of the grabbing point and the grabbing rotation angle. The conditions that need to be met are(10)gtθ−gpθ≤30,gp∩gtgp∪gt>0.25,where gp represents the grab point grab box x,y,w,h parameters predicted by the model. gt represents the actual grabbing frame parameters of the image annotation. gpθ indicates that the model predicts the corresponding angle value of the grab rotation angle label of the grab point. gtθ represents the angle value corresponding to the capture rotation angle label of the actual marked capture point in the image.


## 5.2. Evaluation on Target Recognition

In this work, an improved SSD (ISSD) is proposed to perform the target recognition detection and positioning. To verify the effectiveness of our designed method, we first compare this method with other methods. The result is illustrated in Table1.Table 1
Comparison of ISSD and other methods.
MethodIPDNPrecisionRecallmAPPrecisionRecallmAPFaster-RCNN85.481.773.584.380.972.6YOLO89.285.377.887.984.775.9SSD91.387.480.690.486.579.2ISSD (ours)93.489.284.992.188.183.6It can be seen that, compared with other identification detection methods, the method used in this article can obtain the best performance. The precision of 93.4% and 92.1%, recall of 89.2% and 88.1%, mAP of 84.9% and 83.6% can be obtained on the two datasets. Compared with traditional SSD, the improved algorithm can obtain 2.1% and 1.7% gains on precision, 1.8% and 1.6% gains on recall, 4.3% and 4.4% gains on mAP. These data can verify the correctness of our method.As mentioned early, the improved inception structure is applied to optimize SSD. To verify the effectiveness of this improvement strategy, this article compares the recognition and detection performance of using improved Inception structure and not using this structure. The result is illustrated in Figure5.Figure 5
Evaluation on improved Inception.Obviously, when the Inception structure is not used, the performance of the network is the lowest. After using this structure, the performance of the model will be improved to a certain extent. Especially after improving the Inception model, the performance will be further improved. This shows the correctness of the combination of the Inception structure and the SSD model in this article, and also proves the effectiveness of the improved strategy proposed in this article for the Inception structure.Besides, in the used loss of SSD, this work combines the traditional loss with repulsion loss. To verify the effectiveness of this optimization strategy for network loss, this article compares the recognition and detection performance of using the combined loss and using the single traditional loss. The result is illustrated in Figure6.Figure 6
Evaluation on improved loss.Obviously, when the repulsion loss is not used, the performance of the network is lower. After using the combined loss, the performance of the model will be improved to a certain extent. This shows the correctness of the combination of the traditional SSD loss with repulsion loss in this article, and also proves the effectiveness of the improved strategy proposed in this article for the loss. This loss can guide the network to extract more efficient feature.Last, when performing target recognition and detection, this paper improves the nonmaximum suppression algorithm, and proposes a nonmaximum suppression weight algorithm (NMSW). To verify the effectiveness of this improvement strategy, this work compares the recognition and detection performance of using NMSM and using the traditional NMS. The result is illustrated in Figure7.Figure 7
Evaluation on NMSW.It can be seen that, compared with the traditional NMS method, the NMSW method proposed in this paper can achieve higher performance. On the two datasets, the mAP of 2.3% and 2.1% are improved, respectively. This verifies the correctness of the proposed NMSW algorithm. This is mainly because NMSW overcomes the shortcomings of traditional NMS methods that are easy to ignore by keeping the highest confidence of the bounding box to make full use of the object information.


## 5.3. Evaluation on Grabbing Positioning

In this paper, the trained grabbing positioning model (GPM) is tested, and the performance of the algorithm model is tested. The test set image is composed of the target data set output by the improved SSD target recognition and detection model in the first stage. The detection accuracy is composed of two parts, one is instance detection (ID). That is, the grabbing targets included in the test set are all grabbing targets that appear in the training set and appear at different positions and angles. This detection performance can be suitable for industrial machines to perform posture detection and positioning of grabbed targets of known target categories. The second is object detection (OD), that is, the capture target categories that appear in the test set are not in the network model training set. This detection performance can judge the versatility and adaptability of a machine. High-precision regional posture detection and grabbing point positioning can also be performed for grabbing targets with new shapes. The comparison result with other methods is illustrated in Table2.Table 2
Comparison of GPM and other methods.
MethodIPDNIDODIDODRTGD88.487.988.187.4HCF90.289.589.688.7SDL92.792.291.590.8MOMG95.394.693.993.2GPM (ours)96.596.295.795.1It can be seen that, compared with other grabbing positioning methods, the method used in this work can obtain the best performance. The instance detection accuracy of 96.5% and 95.7%, object detection accuracy of 96.2% and 95.1% can be obtained on the two datasets. Compared with the best listed method, the proposed algorithm can obtain 1.2% and 1.8% gains on instance detection accuracy, 1.6% and 1.9% gains on object detection accuracy. These data can verify the correctness of our method.As mentioned early, this work will normalize a batch of input feature data before the activation function of the convolutional layer of the inception module branch. Using this method, the input data is normally distributed, and each eigenvalue falls in the standard distribution domain. The input eigenvalues will not fall in the saturation region of the activation function, and the influence of singular values on the distribution of neuron data will be reduced. In order to verify the effectiveness of this strategy, this paper compares the network performance when BN is not used with the network performance when BN is used. The result is shown in Figure8.Figure 8
Evaluation on BN layer.Obviously, with the introduction of the BN layer, the performance of the network has been improved. On the IP dataset, the improvement of the two performance indicators is 2.3% and 2.4% respectively. On the DN dataset, the improvement of the two performance indicators is 1.4% and 1.5% respectively. These data verify the effectiveness and correctness of our proposed strategy.


## 6. Conclusion

The combination of vision technology and industrial machines has become a hot research topic in industrial intelligence. However, the machine still has great limitations when crawling. Affected by the various characteristics of the target object and the complex and changeable environmental factors, the research of machine grabbing still faces severe challenges. For the actual complex working conditions, the target recognition detection effect will cause the machine to be unable to complete the accurate grabbing positioning, thereby affecting the production efficiency. This paper proposes a target recognition and grabbing positioning algorithm based on convolutional neural network, which can realize target recognition, detection, positioning and grabbing tasks. First, by investigating the current mainstream target recognition and detection algorithms, an SSD with both real-time and accuracy is selected. Then improve the network according to the requirements and insert the Inception structure. At the same time optimize its loss function and nonmaximum suppression. The improved recognition rate is higher, the target detection frame is closer to the real situation, and the recognition error is greatly reduced. Secondly, this research proposes an algorithm model for regional posture detection and grabbing positioning. The output of the previous stage of the model is the input, and the target can be detected and grasped. In the network, the posture angle of the grabbing target is classified and output, and the position coordinates of the grabbing point are output by the regression method. Massive experiments have proved that our method can perform efficient target recognition and grabbing positioning.



---

*Source: 4360346-2022-03-29.xml*