# Small Target Detection Algorithm Based on Transfer Learning and Deep Separable Network

Authors: Peng Wang; Haiyan Wang; Xiaoyan Li; Lingling Zhang; Ruohai Di; Zhigang Lv
Journal: Journal of Sensors
 (2021)

---

## Abstract

With the development of deep learning, target detection from vision sensor has achieved high accuracy and efficiency. However, small target detection remains a challenge due to inadequate use of semantic information and detailed texture information of underlying features. To solve the above problems, this paper proposes a small target detection algorithm based on Mask R-CNN model which integrates transfer learning and deep separable network. Firstly, the feature pyramid fusion structure is introduced to enhance the learning effect of low-level and high-level features, especially to strengthen the information channel of low-level feature and meanwhile optimize the feature information of small target. Secondly, the ELU function is used as the activation function to solve the problem that the original activation function disappears in the negative half axis gradient. Finally, a new loss function F-Softmax combined with Focal Loss was adopted to solve the imbalance of positive and negative sample proportions. In this paper, self-made data set is used to carry out experiments, and the experimental results show that the proposed algorithm makes the detection accuracy of small targets reach 66.5%.


---

## Body

## 1. Introduction

In recent years, the aerial image target detection technology based on UAV has become one of the forefront research topics [1–3]. Due to the distance from the target, the UAV aerial images are mostly small- and medium-sized targets. From the perspective of absolute size, a small target is defined as a 32∗32 pixel target. In terms of relative size, if the target occupies 0.1 times the size of the whole picture, it can be considered as a small target [4–6]. The traditional target detection algorithm is easy to cause misdetection and missed detection of small targets in these image processing, and the detection rate of small targets is low, so small target detection is the focus and difficulty in this field [7–9].Small target detection is a very important field in image processing, and it is only in recent years that more and more attention has been paid to the research of small target detection [10, 11]. Different Gaussian methods are using in deep learning algorithms to detect small targets in maritime infrared images. However, due to small imaging area of small targets in infrared images and insignificant target features, traditional Gaussian methods have problems such as high false positive rate in target detection [12–15]. A general band selection algorithm based on high-order cumulant is analyzed and applied the general band of the high-order cumulant to detect the small targets [16]. Although the detection effect was optimized to some extent, it had a strong dependence on the data set, and its robustness was poor. The singular value decomposition technology was applied to the convolution feature compression processing to reduce the calculation and storage requirements of the model, and the multiscale training method was adopted to adapt to the change of the scale of aviation targets, but there was still a large rate of missed detection, and the detection rate was seriously affected [17]. The main reasons for missed detection and false detection are that the target object is not only interfered by the luminance, occlusion, and other factors but also affected by the interference factors such as the small scale of the target and the large scale change, the complex and changeable background, and there are many background objects which are very similar to the target.In this paper, we proposed small target detection algorithm based on migration study and separable network to solve the high rate of false positives, poor robustness, and low detection rate under battlefield environment. The key contributions in our work can be summarized as follows:(1)
In order to strengthen the relationship between the shallow layer and the deep layer, three fusion layers under the idea of the fusion feature pyramid are proposed. The new feature layer obtained by fusion is taken as the input of the next layer to learn the feature extraction. In addition, the deep separable convolutional network is used for feature extraction to reduce the computational load(2)
Adopt exponential linear element (ELU) instead of traditional ReLU activation function. It achieves the effect of BN layer and reduces a lot of computation. At the same time, it is more robust to the noise of input change and has low complexity(3)
An improved Softmax loss function, namely, F-Softmax, is proposed. By introducing angle constraint, the distance between classes can be increased, and the distance within classes can be reduced by strict decision conditions. This will make the classification more accurate. The introduction of key factors can reduce the weight of samples which are easy to classify and meanwhile make the model focus on the samples which are difficult to classify during training. To do this can solve the unbalance problem of positive and negative sample


## 2. Design of Small Target Detection Architecture

### 2.1. Network Structure

The Mask R-CNN [18] model is a pyramid-like structure, but the shallow feature map with a large field of view does not have the detailed information of the deep feature map, and the deep feature map of small field view cannot cover the target information. So the Mask R-CNN is not good for small target detection. Meanwhile, the huge calculation of Mask R-CNN model makes the positioning and classification speeds slow. In order to enhance relationship between shallow features and deep features, this paper proposes Mask R-CNN model fused with feature pyramids, as shown in Figure 1.Figure 1
Overall network structure.In this paper, three layers of Conv7_1, Conv8_1, and Conv9_1 are selected for feature pyramid fusion structure. Conv7_1 is fused with Conv6_2, Conv8_1 is fused with Conv7_2, and Conv9_1 is fused with Conv8_2. Next, the fusion calculation of Conv7_1 and Conv6_2 is taken as an example for analysis as shown in Figure2. The other two fusion methods are the same.Figure 2
Fusion structure.The characteristic in Conv7_1 is fused with the characteristic in Conv6_2. Low level feature Conv6_2 needs to change the number of channels through a1×1 convolutional layer to reduce the dimension of the feature graph. Similarly, for Conv7_1, the number of channels should be changed through a 1×1 convolutional layer to change it into 19×19×256. Then, the image size of Conv7_1 should be expanded twice by using bilinear interpolation algorithm to become 38×38×256. Finally, the low level features and high level features are fused to get a new feature layer. The new feature layer obtained by fusion is taken as the input of the next layer to learn the feature extraction.


### 2.2. Transfer Learning Structure

This paper introduced the transfer learning combined with CNN to propose a remote sensing image target model recognition algorithm based on transfer learning. Among them, the source domain is PASCAL VOC2012 data set of ten type of targets. Source task focused on image classification in the source domain. The target domain refers to the PASCAL VOC2012 data set of five small type of targets.Target task is to classify small targets in the image of the land battlefield. The overall structure of the transfer learning method is shown in Figure3.Figure 3
The overall structure of the transfer learning.The framework of transfer learning used in the algorithm is based on the Mask R-CNN network model, each of which includes multiple convolutionlayers, activation layers, pooling layers, and fully connected layers. The algorithm can be divided into two stages: the preliminary training stage and parameter fine-tuning stage. First, classified training with five small types of targets in VOC2012 data set and get the classification model. Then, under this basis model, classification training is carried out in the other kinds of target image.The network model used in this paper includes 13 convolution layers, 13 activation function layers, and 4 pooling layers. Among the convolution layer, the convolution kernel size is 3, zero complement is 1, and step size is 1. Among the pooling layer, the window size is 2, and step size is 2. The activation function used in the model is ReLU activation function. The full connection layer improves overfitting by using dropout, which randomly sets the neurons in the model to 0 with a 50% probability to reduce the dependence of fixed connections between neurons. The classification layer adopts NMS function. There are 10 categories of labels, and each is predicted as the probability of the corresponding category.


### 2.3. Deeply Separable Network

Using the concept of Xception model to balance the accuracy and speed and meanwhile realize extract the attention feature of the image. In the template feature extraction network and feature extraction to be detected network, using depth separable convolution instead of traditional convolution kernels, which means to build the DS-AlexNet (Depthwise Separable-AlexNet) network. And the other module of the network is not need to be changed. To do this can reduce the cost of the network parameters while not affect the accuracy of model.The standard convolution structure is shown in Figure4. Separable convolution is a one-dimensional channel convolution kernel and a two-dimensional position convolution kernel. The channel information and position information in the image are, respectively, learned. The separable convolution structure is shown in Figure 5.Figure 4
Standard convolution.Figure 5
Separable convolution.The main purpose of using separable convolution is to separate the spatial cross-correlation information from the channel cross-correlation information, so as to improve the recognition rate while speeding up calculation. Assume the input feature size isDk×Dk×M, M is the number of input channels. The standard convolution kernel is D×D×N, where D is the length and width of the convolution kernel and N is the number of output channels. The calculation amount of a standard convolution is shown in Equation (1).
(1)Dk×Dk×M×N×D×D.In the case of separable convolution,D×D filters are applied to M input channels, i.e., D×D×M×DK×DK, and N1×1×M convolution filters are applied to combine M input channels into N output channels, i.e., M×N×DK×DK. Merge each value in the 1×1×M feature graph together, and the calculation amount is shown in Equation (2).
(2)D×D×M×1×Dk×Dk+1×1×M×N×Dk×Dk.Compared with the standard convolution structure, such a separable convolution structure requires less computation, as shown in Equation (3).
(3)D×D×M×1×Dk×Dk+1×1×M×N×Dk×DkD×D×M×N×Dk×Dk=1N+1D2.Taking Conv4 to Con5 as an example, the feature mapping of the input is13×13×384, and the standard convolution kernel is 3×3×256.
(4)13×13×384×1×3×3+1×1×256×384×3×313×13×384×256×3×3≈1128.As shown in Equation (4), the amount of calculation is reduced to 1/128 of the original amount.Through the above analysis, it is proved that using the deep separable convolution structure instead of the traditional convolution structure can speed up the calculation and reduce the used computing resources under the condition of ensuring the same feature extraction effect.


### 2.4. Activation Function and Loss Function

#### 2.4.1. Activation Function

The exponential linear unit (ELU) is used to replace the traditional ReLU activation function, and the ELU function expression is shown in Equation (5).
(5)fx=xifx≥0,αex−1,ifx<0,f′x=1ifx≥0,fx+α,ifx<0.The ELU function is an improvement over ReLU. When the parameters are greater than or equal to 0, the computational complexity is low, and the learning speed is fast without the need for exponential operation, which also increases the nonlinear characteristics of the model. When the parameter is less than 0, a smooth function is used instead of the original identity 0, so that the average output value of the activation function is close to zero; therefore, the convergence speed is faster. The BN layer effect is achieved, and a lot of computation is reduced. At the same time, it is more robust to the input noise and has lower complexity.


#### 2.4.2. Loss Function

The classification function used in the Mask R-CNN model is Smooth L1, which approximates the output as a probability distribution, as shown in Equation (6).
(6)Llocx,l,g=∑i∈PostiveN∑m∈cx,cy,w,hxijksmoothL1lim−g^jm.This paper proposes an improved Softmax loss function with a period of increasing angle constraints and a key factor named F-Softmax.(1)
Angle constraintsThe output of the model isw1x,w2x⋯wnx; the classification estimation probability obtained after the loss function is shown in Equation (7).
(7)hwx=1ew1x+ew2x+⋯+ewnxew1xew2x⋯ewnx.x in formula (7) represents the sample, n represents the category, and ewnx represents the parameter weight vector of different sample types. If the input sample x belongs to the n category, then the value of ewnx is the largest; that is, wnx is also required to be the largest. Expand the equation to obtain Equation (8).
(8)wnxcosθn>w1xcosθ1,w2xcosθ2⋯.θ is the included angle between the sample probability vector and the parameter weight vector. Assuming an integer n, Equation (9) can be obtained according to the properties of cos function.
(9)wnxcosθn>w1xcosθ1,w2xcosθ2⋯whereθ1,θ2∈0,πn.The inclusion of the limiting conditions in the formula makes the discrimination more strict, so that in the original loss function, if there isA class of targets that may belong to class A or may belong to class B. At this time, when judging the category of the target, not only the probability vector is required to be the same as the parameter weight vector but also the constraint condition of an included angle is added. Strict criteria can make the distance between classes larger and the distance within classes smaller, so that the classification is more accurate.
(2)
Key factorsAngle constraint considers that the distance between the class and class does not take into account the balance of positive and negative samples, on the basis of the previous section introduced the Focal Loss of ideological building loss function, the Focal Loss formula such as type of (10).
(10)LossFL=−αt1−ptγlogpt,wherept is equal to (11).
(11)Pt=p,ify=1,1−p,else.αt is called the weighting factor; y is called the key factor; 1−pt represents the probability of belonging to the label, within the range of [0,1]; and P is the probability of the target predicted by the model, within the range of [0,1].Finally, the loss function formula is substituted into Focal Loss, which is the proposed loss function formula F-Softmax, as shown in Equation (12).
(12)LFS=1N∑i−αt1−ptγlogp.The optimal initial value selection is given through experiments, andα=0.25 and γ=2 are set. Where n is the number of categories and p is the probability precalculated by Softmax function, the calculation formula is shown in Equation (13).
(13)p=exiwicosθnexiwicosθn+∑i≠yiexiwicosθn,θ∈0,πn.


## 2.1. Network Structure

The Mask R-CNN [18] model is a pyramid-like structure, but the shallow feature map with a large field of view does not have the detailed information of the deep feature map, and the deep feature map of small field view cannot cover the target information. So the Mask R-CNN is not good for small target detection. Meanwhile, the huge calculation of Mask R-CNN model makes the positioning and classification speeds slow. In order to enhance relationship between shallow features and deep features, this paper proposes Mask R-CNN model fused with feature pyramids, as shown in Figure 1.Figure 1
Overall network structure.In this paper, three layers of Conv7_1, Conv8_1, and Conv9_1 are selected for feature pyramid fusion structure. Conv7_1 is fused with Conv6_2, Conv8_1 is fused with Conv7_2, and Conv9_1 is fused with Conv8_2. Next, the fusion calculation of Conv7_1 and Conv6_2 is taken as an example for analysis as shown in Figure2. The other two fusion methods are the same.Figure 2
Fusion structure.The characteristic in Conv7_1 is fused with the characteristic in Conv6_2. Low level feature Conv6_2 needs to change the number of channels through a1×1 convolutional layer to reduce the dimension of the feature graph. Similarly, for Conv7_1, the number of channels should be changed through a 1×1 convolutional layer to change it into 19×19×256. Then, the image size of Conv7_1 should be expanded twice by using bilinear interpolation algorithm to become 38×38×256. Finally, the low level features and high level features are fused to get a new feature layer. The new feature layer obtained by fusion is taken as the input of the next layer to learn the feature extraction.


## 2.2. Transfer Learning Structure

This paper introduced the transfer learning combined with CNN to propose a remote sensing image target model recognition algorithm based on transfer learning. Among them, the source domain is PASCAL VOC2012 data set of ten type of targets. Source task focused on image classification in the source domain. The target domain refers to the PASCAL VOC2012 data set of five small type of targets.Target task is to classify small targets in the image of the land battlefield. The overall structure of the transfer learning method is shown in Figure3.Figure 3
The overall structure of the transfer learning.The framework of transfer learning used in the algorithm is based on the Mask R-CNN network model, each of which includes multiple convolutionlayers, activation layers, pooling layers, and fully connected layers. The algorithm can be divided into two stages: the preliminary training stage and parameter fine-tuning stage. First, classified training with five small types of targets in VOC2012 data set and get the classification model. Then, under this basis model, classification training is carried out in the other kinds of target image.The network model used in this paper includes 13 convolution layers, 13 activation function layers, and 4 pooling layers. Among the convolution layer, the convolution kernel size is 3, zero complement is 1, and step size is 1. Among the pooling layer, the window size is 2, and step size is 2. The activation function used in the model is ReLU activation function. The full connection layer improves overfitting by using dropout, which randomly sets the neurons in the model to 0 with a 50% probability to reduce the dependence of fixed connections between neurons. The classification layer adopts NMS function. There are 10 categories of labels, and each is predicted as the probability of the corresponding category.


## 2.3. Deeply Separable Network

Using the concept of Xception model to balance the accuracy and speed and meanwhile realize extract the attention feature of the image. In the template feature extraction network and feature extraction to be detected network, using depth separable convolution instead of traditional convolution kernels, which means to build the DS-AlexNet (Depthwise Separable-AlexNet) network. And the other module of the network is not need to be changed. To do this can reduce the cost of the network parameters while not affect the accuracy of model.The standard convolution structure is shown in Figure4. Separable convolution is a one-dimensional channel convolution kernel and a two-dimensional position convolution kernel. The channel information and position information in the image are, respectively, learned. The separable convolution structure is shown in Figure 5.Figure 4
Standard convolution.Figure 5
Separable convolution.The main purpose of using separable convolution is to separate the spatial cross-correlation information from the channel cross-correlation information, so as to improve the recognition rate while speeding up calculation. Assume the input feature size isDk×Dk×M, M is the number of input channels. The standard convolution kernel is D×D×N, where D is the length and width of the convolution kernel and N is the number of output channels. The calculation amount of a standard convolution is shown in Equation (1).
(1)Dk×Dk×M×N×D×D.In the case of separable convolution,D×D filters are applied to M input channels, i.e., D×D×M×DK×DK, and N1×1×M convolution filters are applied to combine M input channels into N output channels, i.e., M×N×DK×DK. Merge each value in the 1×1×M feature graph together, and the calculation amount is shown in Equation (2).
(2)D×D×M×1×Dk×Dk+1×1×M×N×Dk×Dk.Compared with the standard convolution structure, such a separable convolution structure requires less computation, as shown in Equation (3).
(3)D×D×M×1×Dk×Dk+1×1×M×N×Dk×DkD×D×M×N×Dk×Dk=1N+1D2.Taking Conv4 to Con5 as an example, the feature mapping of the input is13×13×384, and the standard convolution kernel is 3×3×256.
(4)13×13×384×1×3×3+1×1×256×384×3×313×13×384×256×3×3≈1128.As shown in Equation (4), the amount of calculation is reduced to 1/128 of the original amount.Through the above analysis, it is proved that using the deep separable convolution structure instead of the traditional convolution structure can speed up the calculation and reduce the used computing resources under the condition of ensuring the same feature extraction effect.


## 2.4. Activation Function and Loss Function

### 2.4.1. Activation Function

The exponential linear unit (ELU) is used to replace the traditional ReLU activation function, and the ELU function expression is shown in Equation (5).
(5)fx=xifx≥0,αex−1,ifx<0,f′x=1ifx≥0,fx+α,ifx<0.The ELU function is an improvement over ReLU. When the parameters are greater than or equal to 0, the computational complexity is low, and the learning speed is fast without the need for exponential operation, which also increases the nonlinear characteristics of the model. When the parameter is less than 0, a smooth function is used instead of the original identity 0, so that the average output value of the activation function is close to zero; therefore, the convergence speed is faster. The BN layer effect is achieved, and a lot of computation is reduced. At the same time, it is more robust to the input noise and has lower complexity.


### 2.4.2. Loss Function

The classification function used in the Mask R-CNN model is Smooth L1, which approximates the output as a probability distribution, as shown in Equation (6).
(6)Llocx,l,g=∑i∈PostiveN∑m∈cx,cy,w,hxijksmoothL1lim−g^jm.This paper proposes an improved Softmax loss function with a period of increasing angle constraints and a key factor named F-Softmax.(1)
Angle constraintsThe output of the model isw1x,w2x⋯wnx; the classification estimation probability obtained after the loss function is shown in Equation (7).
(7)hwx=1ew1x+ew2x+⋯+ewnxew1xew2x⋯ewnx.x in formula (7) represents the sample, n represents the category, and ewnx represents the parameter weight vector of different sample types. If the input sample x belongs to the n category, then the value of ewnx is the largest; that is, wnx is also required to be the largest. Expand the equation to obtain Equation (8).
(8)wnxcosθn>w1xcosθ1,w2xcosθ2⋯.θ is the included angle between the sample probability vector and the parameter weight vector. Assuming an integer n, Equation (9) can be obtained according to the properties of cos function.
(9)wnxcosθn>w1xcosθ1,w2xcosθ2⋯whereθ1,θ2∈0,πn.The inclusion of the limiting conditions in the formula makes the discrimination more strict, so that in the original loss function, if there isA class of targets that may belong to class A or may belong to class B. At this time, when judging the category of the target, not only the probability vector is required to be the same as the parameter weight vector but also the constraint condition of an included angle is added. Strict criteria can make the distance between classes larger and the distance within classes smaller, so that the classification is more accurate.
(2)
Key factorsAngle constraint considers that the distance between the class and class does not take into account the balance of positive and negative samples, on the basis of the previous section introduced the Focal Loss of ideological building loss function, the Focal Loss formula such as type of (10).
(10)LossFL=−αt1−ptγlogpt,wherept is equal to (11).
(11)Pt=p,ify=1,1−p,else.αt is called the weighting factor; y is called the key factor; 1−pt represents the probability of belonging to the label, within the range of [0,1]; and P is the probability of the target predicted by the model, within the range of [0,1].Finally, the loss function formula is substituted into Focal Loss, which is the proposed loss function formula F-Softmax, as shown in Equation (12).
(12)LFS=1N∑i−αt1−ptγlogp.The optimal initial value selection is given through experiments, andα=0.25 and γ=2 are set. Where n is the number of categories and p is the probability precalculated by Softmax function, the calculation formula is shown in Equation (13).
(13)p=exiwicosθnexiwicosθn+∑i≠yiexiwicosθn,θ∈0,πn.


## 2.4.1. Activation Function

The exponential linear unit (ELU) is used to replace the traditional ReLU activation function, and the ELU function expression is shown in Equation (5).
(5)fx=xifx≥0,αex−1,ifx<0,f′x=1ifx≥0,fx+α,ifx<0.The ELU function is an improvement over ReLU. When the parameters are greater than or equal to 0, the computational complexity is low, and the learning speed is fast without the need for exponential operation, which also increases the nonlinear characteristics of the model. When the parameter is less than 0, a smooth function is used instead of the original identity 0, so that the average output value of the activation function is close to zero; therefore, the convergence speed is faster. The BN layer effect is achieved, and a lot of computation is reduced. At the same time, it is more robust to the input noise and has lower complexity.


## 2.4.2. Loss Function

The classification function used in the Mask R-CNN model is Smooth L1, which approximates the output as a probability distribution, as shown in Equation (6).
(6)Llocx,l,g=∑i∈PostiveN∑m∈cx,cy,w,hxijksmoothL1lim−g^jm.This paper proposes an improved Softmax loss function with a period of increasing angle constraints and a key factor named F-Softmax.(1)
Angle constraintsThe output of the model isw1x,w2x⋯wnx; the classification estimation probability obtained after the loss function is shown in Equation (7).
(7)hwx=1ew1x+ew2x+⋯+ewnxew1xew2x⋯ewnx.x in formula (7) represents the sample, n represents the category, and ewnx represents the parameter weight vector of different sample types. If the input sample x belongs to the n category, then the value of ewnx is the largest; that is, wnx is also required to be the largest. Expand the equation to obtain Equation (8).
(8)wnxcosθn>w1xcosθ1,w2xcosθ2⋯.θ is the included angle between the sample probability vector and the parameter weight vector. Assuming an integer n, Equation (9) can be obtained according to the properties of cos function.
(9)wnxcosθn>w1xcosθ1,w2xcosθ2⋯whereθ1,θ2∈0,πn.The inclusion of the limiting conditions in the formula makes the discrimination more strict, so that in the original loss function, if there isA class of targets that may belong to class A or may belong to class B. At this time, when judging the category of the target, not only the probability vector is required to be the same as the parameter weight vector but also the constraint condition of an included angle is added. Strict criteria can make the distance between classes larger and the distance within classes smaller, so that the classification is more accurate.
(2)
Key factorsAngle constraint considers that the distance between the class and class does not take into account the balance of positive and negative samples, on the basis of the previous section introduced the Focal Loss of ideological building loss function, the Focal Loss formula such as type of (10).
(10)LossFL=−αt1−ptγlogpt,wherept is equal to (11).
(11)Pt=p,ify=1,1−p,else.αt is called the weighting factor; y is called the key factor; 1−pt represents the probability of belonging to the label, within the range of [0,1]; and P is the probability of the target predicted by the model, within the range of [0,1].Finally, the loss function formula is substituted into Focal Loss, which is the proposed loss function formula F-Softmax, as shown in Equation (12).
(12)LFS=1N∑i−αt1−ptγlogp.The optimal initial value selection is given through experiments, andα=0.25 and γ=2 are set. Where n is the number of categories and p is the probability precalculated by Softmax function, the calculation formula is shown in Equation (13).
(13)p=exiwicosθnexiwicosθn+∑i≠yiexiwicosθn,θ∈0,πn.


## 3. Experimental Results

### 3.1. Introduction to the Experimental Environment

In the Ubuntu 16.04 operating system, the algorithm in this paper adopts the deep learning framework PyTorch to realize the ground-field target detection algorithm based on the multilevel feature pyramid. The experimental platform uses CPU: Intel(R) Core(TM) I5-8600 3.10 GHz; Memory: 16 G; GPU: NVIDIA GTX 1080TI, training and testing the network in the above environment. In order to verify the accuracy and real-time performance of the algorithm, YOLO v3, Faster R-CNN, and Mask R-CNN algorithms with better current performance were selected for comparison, all of which were tested in the same environment. The training set is made by randomly extracting 70% data from the data set, while the test set is made by randomly extracting 30% data from the data set.


### 3.2. Introduction to the Experimental Data Set

The target included tank, person, gun, cannon, helicopter, and car. The data set contains 9,000 images of the abovementioned target. Then, the data was expanded to 27,000 by adding noise and scaling to some extent. We also found 3000 relevant video images containing the target from the network. So the data set consists of total 30,000 images. Each image is manually annotated in accordance with the format of PASCAL VOC data set. Some images of the data set are shown in Figure6.Figure 6
Part of the data set.


### 3.3. Ablation Experiment

The experimental data set is expanded self-made data set. The data set was taken as input, and the parameters of network training were set as follows: learning rate 0.1, the learning rate decreased by 1 order of magnitude after each epoch, regular termf=0.1, and BatchSize 100. After each epoch, the data set was rearranged randomly.


#### 3.3.1. Activation Function

The recognition accuracy of the network on the test set-the iteration step curve (acc-step) and the loss function value-the iteration step curve (loss-step) are shown in Figures7(a) and 7(b). It can be seen that with the update of iteration step, the overall recognition accuracy finally reached more than 90%, and the curves of the identification accuracy and the loss function value of the data set basically leveled off after about 3300 iterations.Figure 7
(a) Acc-step curve. (b) Loss-step curve.
(a)(b)The activation functions in the residual block are, respectively, set to ReLU and LReLU. The activation functions are shown in Equations (5), (14), and (15). Only the activation function in the model is changed; other parts of the model remain unchanged and are trained under the same training set.a is the adjustment parameter, and it control the activation of the ELU function in the negative half axis.
(14)ReLUx=x,ifx>0,0,fx≤0,(15)LReLux=xi,ifxi≥0,αixi,ifxi<0.ai is fixed. i means different channels correspond to different ai.It can be seen from Figure8 that the convergence of LReLU is close to ELU during 20000-25000 iterations. However, compared with ReLU and LReLU, as the number of iterations increases, ELU has the minimum final loss function value, and the training effect of the model is better.Figure 8
Loss function changes.


#### 3.3.2. Comparison of Loss Functions

In order to verify the superiority of F-Softmax function, ROC curve is used to evaluate the influence of various loss functions on the classification of model samples. Softmax loss function and cross-entropy loss function are used to compare with F-Softmax function. In order to ensure objectivity, other parameters of the model remain unchanged.From Figure9, the classification effect of the Softmax is the worst. The cross-entropy loss function can effectively solves the probability problem of multiple classifications and improves the classifier effect. The F-Softmax function not only effectively solves the guidance problem of difficult samples and simple samples but also effectively deals with the problem of sample imbalance, making the loss function more reasonable. The classification model using the loss function has the best classification performance.Figure 9
ROC curve.


#### 3.3.3. Comparative Experiment of Transfer Learning

In order to verify the applicability of the migration study, this paper is the first on the five types of self-made data set to train and get classification model. Then, using this model to all the 10 kinds of target image data set to train the model, the model will give recognition rate and loss function change curve of the two training condition, respectively. The five types of data sets used to train the original model include person, armoured vehicles, gun, tank, and drone. The other five data sets used to verify transfer learning include knives, helicopter, car, bulldozer, and cannon, and the other 10 data sets include all of the above targets. Figure10 shows the comparison of classification accuracy and loss function curves of the model under zero-based learning and transfer learning modes.Figure 10
Comparison of experimental results of transfer learning and zero-based learning.
(a)
5 types of target accuracy-iteration step curve(b)
5 types of objective loss function-iteration step curve(c)
10 types of target correct rate-iteration step curve(d)
10 types of objective loss function-iteration step curveThe parameters of the network model were set as follows: the learning rate was 0.1, the regular termf=0.1, and the BatchSize was 100. After each epoch, the data sets were rearranged randomly, and the amplified self-made data set was used as input to train the network. As can be seen from Figures 10(a) and 10(b), when the five types of data sets are classified, the initial value of classification accuracy of zero-based learning is 11.6% while the transfer learning method in the same period is as high as 62.3%. After about 1800 steps, the accuracy of network classification in the transfer learning mode reached a peak of more than 90%, and after about 4500 steps, the accuracy curve had no obvious change. The classification accuracy and loss function curves of the model combining the transfer learning on all 10 types of targets are shown in Figures 10(c) and 10(d). The initial accuracy values are 21.3% and 44.1%, respectively. After about 2800 steps, the classification accuracy in the transfer learning mode was higher than 92%, slightly higher than the 90% in the zero-basic learning mode of 4800 steps. The iteration curves of transfer learning mode in the above two training sets are smoother, and the model training speed is faster.


#### 3.3.4. Model Comparison Experiment

In this section, the existing YOLO-v3, Faster R-CNN [19], and Mask R-CNN methods will be used to compare with our proposed algorithm which called Ours+. Ours+ is a model for transferring 5 types of data training. All methods are trained in the same data set and tested in the same test set. There are eight parameters used to compare the performance of the four algorithms: mAP (mean Average Precision), AP1 (Average Precision of tank), AP2 (Average Precision of person), AP3 (Average Precision of gun), AP4 (Average Precision of cannon), AP5 (Average Precision of helicopter), AP6 (Average Precision of car), and FPS (Frame Per Second).As shown in Table1, compared with YOLO v3 model, as the method mainly focuses on lightweight detection so it is the fastest among the four models compared in terms of FPS, but the detection accuracy is far behind the other methods. The original design intention of Faster R-CNN and Mask R-CNN is two-stage structure, which have candidate region generation network. So the network is far more complex than other methods, and the detection accuracy is super than YOLO v3. The proposed method improves several shortcomings of Mask R-CNN model, so the accuracy of the FPS is super than Mask R-CNN. Some test results are shown in Figure 11.Table 1
Test data.
Ours+YOLO v3Faster-RCNNMask R-CNNmAP66.559.465.766.0AP168.361.266.768.0AP265.557.365.365.1AP365.356.565.064.5AP466.259.064.865.0AP567.061.566.366.9AP666.760.966.166.5FPS14351512Figure 11
Part of the test results.


## 3.1. Introduction to the Experimental Environment

In the Ubuntu 16.04 operating system, the algorithm in this paper adopts the deep learning framework PyTorch to realize the ground-field target detection algorithm based on the multilevel feature pyramid. The experimental platform uses CPU: Intel(R) Core(TM) I5-8600 3.10 GHz; Memory: 16 G; GPU: NVIDIA GTX 1080TI, training and testing the network in the above environment. In order to verify the accuracy and real-time performance of the algorithm, YOLO v3, Faster R-CNN, and Mask R-CNN algorithms with better current performance were selected for comparison, all of which were tested in the same environment. The training set is made by randomly extracting 70% data from the data set, while the test set is made by randomly extracting 30% data from the data set.


## 3.2. Introduction to the Experimental Data Set

The target included tank, person, gun, cannon, helicopter, and car. The data set contains 9,000 images of the abovementioned target. Then, the data was expanded to 27,000 by adding noise and scaling to some extent. We also found 3000 relevant video images containing the target from the network. So the data set consists of total 30,000 images. Each image is manually annotated in accordance with the format of PASCAL VOC data set. Some images of the data set are shown in Figure6.Figure 6
Part of the data set.


## 3.3. Ablation Experiment

The experimental data set is expanded self-made data set. The data set was taken as input, and the parameters of network training were set as follows: learning rate 0.1, the learning rate decreased by 1 order of magnitude after each epoch, regular termf=0.1, and BatchSize 100. After each epoch, the data set was rearranged randomly.


### 3.3.1. Activation Function

The recognition accuracy of the network on the test set-the iteration step curve (acc-step) and the loss function value-the iteration step curve (loss-step) are shown in Figures7(a) and 7(b). It can be seen that with the update of iteration step, the overall recognition accuracy finally reached more than 90%, and the curves of the identification accuracy and the loss function value of the data set basically leveled off after about 3300 iterations.Figure 7
(a) Acc-step curve. (b) Loss-step curve.
(a)(b)The activation functions in the residual block are, respectively, set to ReLU and LReLU. The activation functions are shown in Equations (5), (14), and (15). Only the activation function in the model is changed; other parts of the model remain unchanged and are trained under the same training set.a is the adjustment parameter, and it control the activation of the ELU function in the negative half axis.
(14)ReLUx=x,ifx>0,0,fx≤0,(15)LReLux=xi,ifxi≥0,αixi,ifxi<0.ai is fixed. i means different channels correspond to different ai.It can be seen from Figure8 that the convergence of LReLU is close to ELU during 20000-25000 iterations. However, compared with ReLU and LReLU, as the number of iterations increases, ELU has the minimum final loss function value, and the training effect of the model is better.Figure 8
Loss function changes.


### 3.3.2. Comparison of Loss Functions

In order to verify the superiority of F-Softmax function, ROC curve is used to evaluate the influence of various loss functions on the classification of model samples. Softmax loss function and cross-entropy loss function are used to compare with F-Softmax function. In order to ensure objectivity, other parameters of the model remain unchanged.From Figure9, the classification effect of the Softmax is the worst. The cross-entropy loss function can effectively solves the probability problem of multiple classifications and improves the classifier effect. The F-Softmax function not only effectively solves the guidance problem of difficult samples and simple samples but also effectively deals with the problem of sample imbalance, making the loss function more reasonable. The classification model using the loss function has the best classification performance.Figure 9
ROC curve.


### 3.3.3. Comparative Experiment of Transfer Learning

In order to verify the applicability of the migration study, this paper is the first on the five types of self-made data set to train and get classification model. Then, using this model to all the 10 kinds of target image data set to train the model, the model will give recognition rate and loss function change curve of the two training condition, respectively. The five types of data sets used to train the original model include person, armoured vehicles, gun, tank, and drone. The other five data sets used to verify transfer learning include knives, helicopter, car, bulldozer, and cannon, and the other 10 data sets include all of the above targets. Figure10 shows the comparison of classification accuracy and loss function curves of the model under zero-based learning and transfer learning modes.Figure 10
Comparison of experimental results of transfer learning and zero-based learning.
(a)
5 types of target accuracy-iteration step curve(b)
5 types of objective loss function-iteration step curve(c)
10 types of target correct rate-iteration step curve(d)
10 types of objective loss function-iteration step curveThe parameters of the network model were set as follows: the learning rate was 0.1, the regular termf=0.1, and the BatchSize was 100. After each epoch, the data sets were rearranged randomly, and the amplified self-made data set was used as input to train the network. As can be seen from Figures 10(a) and 10(b), when the five types of data sets are classified, the initial value of classification accuracy of zero-based learning is 11.6% while the transfer learning method in the same period is as high as 62.3%. After about 1800 steps, the accuracy of network classification in the transfer learning mode reached a peak of more than 90%, and after about 4500 steps, the accuracy curve had no obvious change. The classification accuracy and loss function curves of the model combining the transfer learning on all 10 types of targets are shown in Figures 10(c) and 10(d). The initial accuracy values are 21.3% and 44.1%, respectively. After about 2800 steps, the classification accuracy in the transfer learning mode was higher than 92%, slightly higher than the 90% in the zero-basic learning mode of 4800 steps. The iteration curves of transfer learning mode in the above two training sets are smoother, and the model training speed is faster.


### 3.3.4. Model Comparison Experiment

In this section, the existing YOLO-v3, Faster R-CNN [19], and Mask R-CNN methods will be used to compare with our proposed algorithm which called Ours+. Ours+ is a model for transferring 5 types of data training. All methods are trained in the same data set and tested in the same test set. There are eight parameters used to compare the performance of the four algorithms: mAP (mean Average Precision), AP1 (Average Precision of tank), AP2 (Average Precision of person), AP3 (Average Precision of gun), AP4 (Average Precision of cannon), AP5 (Average Precision of helicopter), AP6 (Average Precision of car), and FPS (Frame Per Second).As shown in Table1, compared with YOLO v3 model, as the method mainly focuses on lightweight detection so it is the fastest among the four models compared in terms of FPS, but the detection accuracy is far behind the other methods. The original design intention of Faster R-CNN and Mask R-CNN is two-stage structure, which have candidate region generation network. So the network is far more complex than other methods, and the detection accuracy is super than YOLO v3. The proposed method improves several shortcomings of Mask R-CNN model, so the accuracy of the FPS is super than Mask R-CNN. Some test results are shown in Figure 11.Table 1
Test data.
Ours+YOLO v3Faster-RCNNMask R-CNNmAP66.559.465.766.0AP168.361.266.768.0AP265.557.365.365.1AP365.356.565.064.5AP466.259.064.865.0AP567.061.566.366.9AP666.760.966.166.5FPS14351512Figure 11
Part of the test results.


## 3.3.1. Activation Function

The recognition accuracy of the network on the test set-the iteration step curve (acc-step) and the loss function value-the iteration step curve (loss-step) are shown in Figures7(a) and 7(b). It can be seen that with the update of iteration step, the overall recognition accuracy finally reached more than 90%, and the curves of the identification accuracy and the loss function value of the data set basically leveled off after about 3300 iterations.Figure 7
(a) Acc-step curve. (b) Loss-step curve.
(a)(b)The activation functions in the residual block are, respectively, set to ReLU and LReLU. The activation functions are shown in Equations (5), (14), and (15). Only the activation function in the model is changed; other parts of the model remain unchanged and are trained under the same training set.a is the adjustment parameter, and it control the activation of the ELU function in the negative half axis.
(14)ReLUx=x,ifx>0,0,fx≤0,(15)LReLux=xi,ifxi≥0,αixi,ifxi<0.ai is fixed. i means different channels correspond to different ai.It can be seen from Figure8 that the convergence of LReLU is close to ELU during 20000-25000 iterations. However, compared with ReLU and LReLU, as the number of iterations increases, ELU has the minimum final loss function value, and the training effect of the model is better.Figure 8
Loss function changes.


## 3.3.2. Comparison of Loss Functions

In order to verify the superiority of F-Softmax function, ROC curve is used to evaluate the influence of various loss functions on the classification of model samples. Softmax loss function and cross-entropy loss function are used to compare with F-Softmax function. In order to ensure objectivity, other parameters of the model remain unchanged.From Figure9, the classification effect of the Softmax is the worst. The cross-entropy loss function can effectively solves the probability problem of multiple classifications and improves the classifier effect. The F-Softmax function not only effectively solves the guidance problem of difficult samples and simple samples but also effectively deals with the problem of sample imbalance, making the loss function more reasonable. The classification model using the loss function has the best classification performance.Figure 9
ROC curve.


## 3.3.3. Comparative Experiment of Transfer Learning

In order to verify the applicability of the migration study, this paper is the first on the five types of self-made data set to train and get classification model. Then, using this model to all the 10 kinds of target image data set to train the model, the model will give recognition rate and loss function change curve of the two training condition, respectively. The five types of data sets used to train the original model include person, armoured vehicles, gun, tank, and drone. The other five data sets used to verify transfer learning include knives, helicopter, car, bulldozer, and cannon, and the other 10 data sets include all of the above targets. Figure10 shows the comparison of classification accuracy and loss function curves of the model under zero-based learning and transfer learning modes.Figure 10
Comparison of experimental results of transfer learning and zero-based learning.
(a)
5 types of target accuracy-iteration step curve(b)
5 types of objective loss function-iteration step curve(c)
10 types of target correct rate-iteration step curve(d)
10 types of objective loss function-iteration step curveThe parameters of the network model were set as follows: the learning rate was 0.1, the regular termf=0.1, and the BatchSize was 100. After each epoch, the data sets were rearranged randomly, and the amplified self-made data set was used as input to train the network. As can be seen from Figures 10(a) and 10(b), when the five types of data sets are classified, the initial value of classification accuracy of zero-based learning is 11.6% while the transfer learning method in the same period is as high as 62.3%. After about 1800 steps, the accuracy of network classification in the transfer learning mode reached a peak of more than 90%, and after about 4500 steps, the accuracy curve had no obvious change. The classification accuracy and loss function curves of the model combining the transfer learning on all 10 types of targets are shown in Figures 10(c) and 10(d). The initial accuracy values are 21.3% and 44.1%, respectively. After about 2800 steps, the classification accuracy in the transfer learning mode was higher than 92%, slightly higher than the 90% in the zero-basic learning mode of 4800 steps. The iteration curves of transfer learning mode in the above two training sets are smoother, and the model training speed is faster.


## 3.3.4. Model Comparison Experiment

In this section, the existing YOLO-v3, Faster R-CNN [19], and Mask R-CNN methods will be used to compare with our proposed algorithm which called Ours+. Ours+ is a model for transferring 5 types of data training. All methods are trained in the same data set and tested in the same test set. There are eight parameters used to compare the performance of the four algorithms: mAP (mean Average Precision), AP1 (Average Precision of tank), AP2 (Average Precision of person), AP3 (Average Precision of gun), AP4 (Average Precision of cannon), AP5 (Average Precision of helicopter), AP6 (Average Precision of car), and FPS (Frame Per Second).As shown in Table1, compared with YOLO v3 model, as the method mainly focuses on lightweight detection so it is the fastest among the four models compared in terms of FPS, but the detection accuracy is far behind the other methods. The original design intention of Faster R-CNN and Mask R-CNN is two-stage structure, which have candidate region generation network. So the network is far more complex than other methods, and the detection accuracy is super than YOLO v3. The proposed method improves several shortcomings of Mask R-CNN model, so the accuracy of the FPS is super than Mask R-CNN. Some test results are shown in Figure 11.Table 1
Test data.
Ours+YOLO v3Faster-RCNNMask R-CNNmAP66.559.465.766.0AP168.361.266.768.0AP265.557.365.365.1AP365.356.565.064.5AP466.259.064.865.0AP567.061.566.366.9AP666.760.966.166.5FPS14351512Figure 11
Part of the test results.


## 4. Conclusion

To solve the problem of low accuracy of small target detection, this paper proposes a small target detection algorithm based on transfer learning and deep separable network. Firstly, feature extraction is carried out by deep separable convolutional network, which reduces the amount of computation. Then, the feature pyramid fusion structure is used to fuse the high-level and low-level feature information, optimize the shallow feature information of the network, and effectively compensate for the loss of information caused by continuous pooling, so as to extract more shallow detail texture information and improve the detection performance of small targets. Finally, the activation function and loss function are optimized to solve the imbalance of positive and negative samples, so as to optimize the network performance. The whole network model is trained by transferring the learning method, and experiments are carried out on the PASCAL VOC2012 data set. The experimental results show that the proposed model is significantly better than other algorithm models in the detection accuracy of small targets.



---

*Source: 9006288-2021-10-04.xml*