SAE‐CenterNet: Self‐attention enhanced CenterNet for small dense object detection

Electronics LettersVolume 59, Issue 3 e212732 LetterOpen Access SAE-CenterNet: Self-attention enhanced CenterNet for small dense object detection Yaoshun Li, Yaoshun Li orcid.org/0000-0003-3180-8435 Hubei Key Laboratory of Intelligent Robot, Wuhan Institute of Technology, Wuhan, China Contribution: Conceptualization, Data curation, ​Investigation, Methodology, Resources, Software, Supervision, Validation, Visualization, Writing - original draftSearch for more papers by this authorLizhi Liu, Lizhi Liu Hubei Key Laboratory of Intelligent Robot, Wuhan Institute of Technology, Wuhan, China Contribution: Formal analysis, Project administrationSearch for more papers by this authorTongwei Lu, Corresponding Author Tongwei Lu [email protected] Hubei Key Laboratory of Intelligent Robot, Wuhan Institute of Technology, Wuhan, China Email: [email protected] Contribution: Funding acquisition, Writing - review & editingSearch for more papers by this author Yaoshun Li, Yaoshun Li orcid.org/0000-0003-3180-8435 Hubei Key Laboratory of Intelligent Robot, Wuhan Institute of Technology, Wuhan, China Contribution: Conceptualization, Data curation, ​Investigation, Methodology, Resources, Software, Supervision, Validation, Visualization, Writing - original draftSearch for more papers by this authorLizhi Liu, Lizhi Liu Hubei Key Laboratory of Intelligent Robot, Wuhan Institute of Technology, Wuhan, China Contribution: Formal analysis, Project administrationSearch for more papers by this authorTongwei Lu, Corresponding Author Tongwei Lu [email protected] Hubei Key Laboratory of Intelligent Robot, Wuhan Institute of Technology, Wuhan, China Email: [email protected] Contribution: Funding acquisition, Writing - review & editingSearch for more papers by this author First published: 01 February 2023 https://doi.org/10.1049/ell2.12732AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract The existing object detection models have not been sufficiently optimized for small dense objects. One of the most common solutions is to extract multi-scale features via feature pyramid network (FPN). However, the information loss of downsampling in multi-scale feature extraction will seriously affect the detection accuracy. Therefore, a dynamic attention convolution (DAC) for downsampling is developed, which can embed regional information for each single pixel. Besides, an attention fusion module (AFM) is also designed to alleviate the data inconsistency from different layers during multi-scale feature fusion. Based on these, the proposed model, SAE-CenterNet, has achieved optimal performance in the mainstream object detection models on the small dense rebar dataset. For example, with 6 FPS decreasing, the m A P 50 $ mAP_{50}$ , m A P 75 $ mAP_{75}$ , R e c a l l 50 $ Recall_{50}$ and R e c a l l 75 $ Recall_{75}$ of SAE-CenterNet is 87.3%, 57.6%, 88.6% and 67.2%, respectively, which are 8.0%, 13.5%, 8.3% and 10.4% higher than the baseline CenterNet, respectively. Introduction The existing object detection models can be divided into anchor-based and anchor-free detectors. Anchor-based detectors are represented by RCNN series [1-4], YOLO series [5-8], SSD series [9, 10], which set a large number of predefined anchor boxes to provide prior knowledge for the models, which can obtain strong performance. However, the redundant negative samples bring from anchor boxes reduces the inference speeds. Anchor-free detectors are represented by CenterNet [11], CornerNet [12], FCOS [13] and DETR [14], which discard the anchor boxes and regard the objects as the key points to predict. Through effective training, anchor-free detectors can achieve the competitive performance to the anchor-based detectors, which have more potential in the generalization. The above detectors are all trained on common dataset such as COCO [15], and have not been optimized for small dense objets. The straightforward way to improve small dense object detection is to design multi-scale feature extraction structures, such as FPN [16] to fully combine the image details of shallow representation with the semantic information of deep representation. However, they cannot alleviate the information loss of the small dense objects in multiple downsampling operations. The existing downsampling operations can not adaptively learn the spatial relationship of local areas, and cannot avoid the information loss in the downsampling process. In addition, deep semantic information and shallow geometric features are combined by element-wise addition, without considering the feature inconsistency between semantic and geometric information. For small dense objects, because the deep feature maps can not obtain the corresponding scale information, which makes the deep feature maps carry a lot of background noises. If such background noises are fused into shallow features, which will interfere with the representation ability of semantic information, make semantic features unable to better guide shallow training, and increase the burden of shallow layers. Deng et al. [17, 18] used super-resolution technology and attention map to reduce the information coupling between feature maps. However, the introduced modules have increased the computation of the model, which is not conducive to terminal deployment. Gong et al. [19] proposed to use statistical methods to search the fusion factors between different layers, so as to enhance the deep semantic information. But the fusion factors rely on large amounts of data and prior knowledge, which improves the training difficulty of the model. In view of the above problems, we propose a self-attention enhanced CenterNet (SAE-CenterNet) for small dense object detection based on CenterNet [11]. By proposing a dynamic attention convolution (DAC) to learn the spatial relationships in the local areas, the information loss is reduced in feature extraction process, and designing an attention fusion module (AFM) to adaptively adjust the features and enhance semantic information from high-level layers in the feature fusion process, thereby effectively improving the accuracy of small dense object detection. To summarize, the contributions of this paper are as follows: (1) Using DAC for downsampling to reduce the feature loss by utilizing self-attention to dynamically aggregate local information on each pixel, thus greatly prolonging the transmission distance of geometric features. (2) Designing AFM to enhance the semantic features from high-level layers in the feature fusion process by learning hard and soft attention weights corresponding to feature maps with different scales, thus reducing the background noise and improve the data consistency in multi-scale feature fusion process. (3) Benefiting from the above improvements, experiments on the small dense rebar dataset show that the proposed SAE-CenterNet reliably improves the performance of small dense object detection. Proposed methods The overall architecture of SAE-CenterNet is shown in Figure 1. SAE-CenterNet uses images with 416 × 416 × 3 $ 416\times 416\times 3$ pixels as the input and ResNet50 [20] as the backbone to obtain a 4-times-downsampling feature map F1 with 104 × 104 × C $ 104\times 104\times C$ pixels. C represents the channel dimension. F1 will be as the input of a multi-scale feature extraction process to obtain three feature maps { F 2 , F 3 , F 4 } $ \lbrace F_2, F_3, F_4\rbrace $ with different scales, so as to further extract the deeper information. We utilize DAC for downsampling, which can alleviate the information loss compared to traditional downsampling operations. The multi-scale feature extraction process can be expressed as follows. F i = R e s N e t 50 ( I n p u t ) i = 1 f C o n v 1 × 1 ( f C o n v 3 × 3 ( ( D A C ( F i − 1 ) ) ) i = 2 , 3 , 4 . $$\begin{equation} F_{i}={\left\lbrace \begin{matrix} ResNet50(Input)&i=1 \\[6pt] f_{Conv}^{1\times 1}(f_{Conv}^{3\times 3}((DAC(F_{i-1})))&i=2,3,4 \end{matrix}\right.}. \end{equation}$$ (1) Fig. 1Open in figure viewerPowerPoint The overall architecture of SAE-CenterNet. DAC is used for downsampling to reduce the feature loss by utilizing self-attention to dynamically aggregate local information on each pixel. AFM is used to enhance the semantic features from high-level layers in the feature fusion process by learning hard and soft attention weights corresponding to feature maps with different scales Among them, I n p u t $ Input$ represents the input images, f C o n v 1 × 1 ( · ) $ f_{Conv}^{1\times 1}(\cdot )$ and f C o n v 3 × 3 ( · ) $ f_{Conv}^{3\times 3}(\cdot )$ represent the convolution with 1 × 1 and 3 × 3 kernel, respectively. In the process of multi-scale feature fusion, we fuse { F 1 , F 2 , F 3 , F 4 } $ \lbrace F_1, F_2, F_3, F_4\rbrace $ from top to bottom, so as to utilize high-level semantic information to assist the low-level structure information to locate objects. To eliminate the feature inconsistency of different layers, we construct AFM to calculate the relevance between different layers and enhance the most relevant semantic information to shallow features in deeper layers, which improves the discrimination ability of model. Three feature maps { G 3 , G 2 , G 1 } $ \lbrace G_3, G_2, G_1\rbrace $ will be output in turn in this stage. The multi-scale feature fusion process can be expressed as follows. G i = A F M ( F i , f C o n v 1 × 1 ( f C o n v 3 × 3 ( f U S 2 ( F i + 1 ) ) ) i = 3 A F M ( F i , f C o n v 1 × 1 ( f C o n v 3 × 3 ( f U S 2 ( G i + 1 ) ) ) i = 1 , 2 . $$\begin{equation} G_{i}= {\left\lbrace \begin{matrix} AFM(F_{i}, f_{Conv}^{1\times 1}(f_{Conv}^{3\times 3}(f_{US}^{2}(F_{i+1})))&i=3 \\[6pt] AFM(F_{i}, f_{Conv}^{1\times 1}(f_{Conv}^{3\times 3}(f_{US}^{2}(G_{i+1})))&i=1,2 \end{matrix}\right.}. \end{equation}$$ (2) Among them, f U S 2 ( · ) $ f_{US}^{2}(\cdot )$ represents the bilinear-interpolation upsampling with 2-times-upsampling factor. G1, which contains rich geometric and context information, will be used for subsequent prediction. The structure of DAC is shown in Figure 1. For the input feature map F e $ Fe$ with H × W × C $ H\times W\times C$ pixels, we use a sliding window of L × L $ L\times L$ size with S stride to select features. The features related to i-th window denotes as F e i $ Fe_{i}$ . Then, we utilize three different convolution with 1 × 1 kernel to encode F e i $ Fe_{i}$ and unfold them, so as to map F e i $ Fe_{i}$ to three vector spaces { α i , β i , γ i } $ \lbrace \alpha _{i}, \beta _{i}, \gamma _{i}\rbrace $ with the same dimension of L 2 × C $ L^2\times C$ pixels. For each α i j ( j ∈ [ 1 , L 2 ] ) $ \alpha _{ij}(j\in [1,L^{2}])$ in α i $ \alpha _{i}$ , β i j ( j ∈ [ 1 , L 2 ] ) $ \beta _{ij}(j\in [1,L^{2}])$ in β i $ \beta _{i}$ , we calculate the relevance r i , j $ r_{i,j}$ between these two vectors by inner product as follows. r i , j = α i j , β i j T . $$\begin{equation} r_{i,j}={\left\langle \alpha _{ij}, \beta _{ij}^T \right\rangle} . \end{equation}$$ (3) We apply the softmax operation on the column dimension of r i $ r_i$ to get the overall attention map A i $ A_i$ . The calculation is as follows. A i = S o f t m a x ( r i ) . $$\begin{equation} A_i = Softmax(r_i). \end{equation}$$ (4) For each A i j ( j ∈ [ 1 , L 2 ] ) $ A_{ij}(j\in [1,L^{2}])$ in A i $ A_{i}$ , γ i j ( j ∈ [ 1 , L 2 ] ) $ \gamma _{ij}(j\in [1,L^{2}])$ in γ i $ \gamma _{i}$ , we use the weight in A i j $ A_{ij}$ to aggregate the pixel features in γ i j $ \gamma _{ij}$ into a global context vector δ i , j $ \delta _{i,j}$ . The calculation is as follows. δ i , j = A i j , γ i j . $$\begin{equation} \delta _{i,j}={\left\langle A_{ij}, \gamma _{ij} \right\rangle} . \end{equation}$$ (5) Compared to γ i , j $ \gamma _{i,j}$ , δ i , j $ \delta _{i,j}$ embeds the global information of the sliding window region. Like the convolutional operation, we sum all the δ i j $ \delta _{ij}$ to obtain the final vector of 1 × C $ 1\times C$ . Existing pooling operation cannot model the spatial information and results in a certain degree of resolution degradation. DAC utilizes sliding windows and self-attention mechanism to aggregate the spatial information of local areas into single pixel, which reduces the information loss in the downsampling process. The structure of AFM is shown in Figure 1. In once AFM, low-level feature map (LF) and high-level feature map (HF) will be encoded to three different vectors { Q , K , V } $ \lbrace Q, K, V\rbrace $ . Q is from LF, K and V are from HF. In order to transfer high-level semantic information to shallow geometric information, we use relevance embedding to embed the relevance between the LF and HF by estimating the similarity between Q and K. Next, Q and K are unfolded into patches, and each patch denotes as q i ( i ∈ [ 1 , H L F × W L F ] ) $ q_i(i\in [1,H_{LF}\times W_{LF}])$ and k j ( j ∈ [ 1 , H H F × W H F ] ) $ k_j(j\in [1,H_{HF}\times W_{HF}])$ . For each patch q i $ q_i$ in Q and k j $ k_j$ in K, we calculate the relevance R i , j $ R_{i,j}$ between these two patches by inner product as follows. R i , j = q i , k j . $$\begin{equation} R_{i,j}={\left\langle q_i, k_j \right\rangle} . \end{equation}$$ (6) The relevance establishes a feature mapping from HF to LF and is further used to obtain the hard attention map H A $ HA$ and the soft attention map S A $ SA$ . H A $ HA$ represents the position indexes of the features most related to Q in V, and S A $ SA$ represents the attention weights corresponding to H A $ HA$ . The hard attention map from the i-th element H A i ( i ∈ [ 1 , H H F × W H F ] ) $ HA_i(i\in [1,H_{HF}\times W_{HF}])$ is calculated from the relevance R i , j $ R_{i,j}$ as follows. H A i = a r g max j R i , j . $$\begin{equation} HA_i=arg\max _{j} R_{i,j}. \end{equation}$$ (7) H A i $HA_i$ represents the most relevant position in the V to the i-th position in the Q. The soft attention map from the i-th element S A i ( i ∈ [ 1 , H H F × W H F ] ) $ SA_i(i\in [1,H_{HF}\times W_{HF}])$ is computed from R i , j $ R_{i,j}$ to represent the confidence of H A i $ HA_i$ . S A i = max j R i , j . $$\begin{equation} SA_i=\max _{j} R_{i,j}. \end{equation}$$ (8) S A i $SA_i$ represents the most relevant feature confidence in the V to the i-th position in the Q. Existing attention mechanism takes a weighted sum of V for each query q i $ q_i$ . However, such an operation may cause smoothing of features in transferring. Therefore, we utilize H A $ HA$ to only transfer the most relevant semantic features from the V to Q for each query q i $ q_i$ . To obtain the transferred semantic features T from V, we apply an index selection operation to V using the H A $ HA$ as the index. T i = V H A i . $$\begin{equation} T_i=V_{HA_i}. \end{equation}$$ (9) T i $T_i$ represents the value of i-th position in T, which is selected from the H A i $ HA_i$ -th position of V. Subsequently, we utilize the S A $ SA$ to synthesize features from HF and LF. During the synthesis process, relevant semantic features should be enhanced while the less relevant ones should be relived. To achieve that, we concatenate T and V in channel dimension to leverage more information from the V. Such fused features are further element-wisely multiplied by the S A $ SA$ to obtain feature map S with enhanced texture information. Finally S is concatenated back to LF to get the final output of feature fusion. This operation can be represented as follows. S = [ V , T ] ⊙ S A , $$\begin{equation} S=[V,T]\odot SA, \end{equation}$$ (10) F o u t = f C o n v 1 × 1 ( [ L F , S ] ) . $$\begin{equation} F_{out}=f_{Conv}^{1\times 1}([LF,S]). \end{equation}$$ (11) F o u t $ F_{out}$ indicates the synthesized output features. ⊙ ( · ) $ \odot (\cdot )$ denotes element-wise multiplication between feature maps. After the above operations, the semantic information from the deep layers can be effectively transferred to shallow geometric information, which boosts a more accurate process of geometric information generation. Experiments We use the small dense rebar dataset from website https://www.datafountain.cn/competitions/332/datasets to conduct experiments. The rebar in the dataset are densely distributed, which is suitable for verifying the validity of SAE-CenterNet. The size of the input image is scaled to 416 × 416 pixels by bicubic interpolation. We use the Pytorch 1.8 to code and deploy the model on Ubuntu 20.04 system. The CPU is i5-9400F with 2.9 GHz, the RAM is 16 GB. The GPU is RTX 3050, and video memory is 8GB. We utilize mAP (mean Average Precision) [15], FPS (Frame Per Second) and Recall [15] to evaluate the detection performance. We accumulate the results by calculating m A P 50 $ mAP_{50}$ , m A P 75 $ mAP_{75}$ , R e c a l l 50 $ Recall_{50}$ and R e c a l l 75 $ Recall_{75}$ in COCO [15]. Among them, m A P i $ mAP_{i}$ and R e c a l l i $ Recall_{i}$ indicate that the detection box is judged as true positive when the IOU value between detection box and GT box is over i 100 $ \frac{i}{100}$ . To verify the effectiveness of the DAC and AFM, we use CenterNet [11] as the baseline to carry out a series of ablation experiments on the rebar test set. The experimental results are shown in Table 1. The m A P 50 $ mAP_{50}$ , m A P 75 $ mAP_{75}$ , R e c a l l 50 $ Recall_{50}$ and R e c a l l 75 $ Recall_{75}$ of SAE-CenterNet is 87.3%, 57.6%, 88.6% and 67.2%, respectively, which are 8.0%, 13.5%, 8.3% and 10.4% higher than the baseline CenterNet [11], respectively. The speed is only slightly decreased (from 43 FPS to 37 FPS). It shows that the modules proposed in this paper can effectively improve the detection accuracy of small dense object detection on the basis of inference speed. Table 1. The ablation experiments for DAC and AFM on the rebar test set CenterNet[11] DAC AFM m A P 50 $ mAP_{50}$ (%) m A P 75 $ mAP_{75}$ (%) R e c a l l 50 $ Recall_{50}$ (%) R e c a l l 75 $ Recall_{75}$ (%) FPS ( f r a m e · s − 1 $ frame \cdot s^{-1}$ ) ✓ $\checkmark$ 79.3 44.1 80.3 56.8 43 ✓ $\checkmark$ ✓ $\checkmark$ 83.3 48.3 85.6 62.4 39 ✓ $\checkmark$ ✓ $\checkmark$ 81.9 45.7 82.4 57.1 41 ✓ $\checkmark$ ✓ $\checkmark$ ✓ $\checkmark$ 87.3 57.6 88.6 67.2 37 Each module improved in this paper has a certain improvement in the detection accuracy. The DAC brings major improvement (4.0% m A P 50 $ mAP_{50}$ , 4.2% m A P 75 $ mAP_{75}$ , 5.3% R e c a l l 50 $ Recall_{50}$ and 5.6% R e c a l l 75 $ Recall_{75}$ ), which assists in enhancing the feature representation in feature extraction. The AFM improves m A P 50 $ mAP_{50}$ by 2.6%, m A P 75 $ mAP_{75}$ by 1.6%, R e c a l l 50 $ Recall_{50}$ by 2.1% and R e c a l l 75 $ Recall_{75}$ by 0.3% compared with the baseline. By enhancing the most relevant semantic features in multi-scale feature fusion, the AFM can adjust adaptively the feature information from different layers and enhance the model's ability to learn locations of small dense objects. Although the two proposed modules both decrease the detection speed, the decrease is small and can be accepted. We randomly select four rebar images with inconsistent direction, light, density, quantity and background in the test set, and test them on baseline CenterNet [11] and SAE-CenterNet. The test results are shown in Figure 2. Fig. 2Open in figure viewerPowerPoint The detection results of CenterNet [11] and SAE-CenterNet. (a) represents the results of CenterNet[11]; (b) represents the results of CenterNet[11] with DAC; (c) represents the results of CenterNet[11] with AFM; (d) represents the results of SAE-CenterNet When DAC is introduced into CenterNet [11], the detection accuracy is greatly improved, which shows that the model learns the spatial relationship between foreground and background through self-attention. The detection results of (c) show that the AFM helps identify the shallow features through deep semantic information, but due to the lack of spatial correlation of local areas, some background detection boxes are not well suppressed. SAE-CenterNet combines the advantages of DAC and AFM and can recall more true positive results, thus improving the detection accuracy. In summary, the actual detection results show that SAE-CenterNet is more capable of small dense object detection tasks under complex conditions than CenterNet [11]. To further explore the performance of the SAE-CenterNet, we select 8 existing object detection models including Faster R-CNN [2], SSD [9], RetinaNet [22], YOLOv3 [7], YOLOv4 [8], YOLOv5M [23], YOLOv5L [23], DETR [14] to test the rebar test set. The comparisons with the performances are shown in Table 2. Table 2. The performances of the different models on the rebar test set Model Backbone m A P 50 $ mAP_{50}$ (%) m A P 75 $ mAP_{75}$ (%) R e c a l l 50 $ Recall_{50}$ (%) R e c a l l 75 $ Recall_{75}$ (%) FPS ( f r a m e · s − 1 $ frame \cdot s^{-1}$ ) Faster R-CNN[2] Resnet50[20] 11.5 11.3 10.9 10.4 8 SSD[9] VGG16[21] 25.7 24.3 22.6 16.5 33 RetinaNet[22] Resnet50[20] 29.0 14.9 29.9 19.8 31 YOLOv3[7] Darknet53[7] 83.1 35.3 84.9 51.0 26 YOLOv4[8] CSPDarknet53[8] 84.9 49.1 86.9 63.6 24 YOLOv5M[23] CSPDarknet53[8] 79.7 41.2 81.9 49.8 25 YOLOv5L[23] CSPDarknet53[8] 80.9 37.7 82.4 48.1 21 DETR[14] Resnet50[20] 3.2 0.1 14.6 1.6 23 SAE-CenterNet Resnet50[20] 87.3 57.6 88.6 67.2 37 The performance of SAE-CenterNet is optimal, and the m A P 50 $ mAP_{50}$ of SAE-CenterNet is 87.3%, which is 75.8%, 61.6%, 58.3%, 4.1%, 2.4%, 7.6%, 6.4%, 84.1% higher than the above models, respectively. SAE-CenterNet not only achieves the best result in precision, but also performs the best in inference speed, reaches 37 FPS. Which proves the effectiveness of the improvements in this paper. Conclusion We proposed SAE-CenterNet to improve the performance of small dense object detection. The proposed DAC and AFM performed well in ablation experiments and greatly elevated the precision of small dense rebar detection. The DAC fuses spatial relationship in the local area by calculating the self-attention in feature extraction process, thus leveraging the feature loss in downsampling. The proposed AFM alleviates the data inconsistence between adjacent feature layers with different scales during feature fusion process. Compared with 8 models, SAE-CenterNet achieves a higher performance on rebar test set. However, some undetected results remain in heavily occluded rebar images, we will continue to improve it with the latest computer technology in the future and make it better applied to actual production. Author contributions Yaoshun Li: Conceptualization, data curation, investigation, methodology, resources, software, supervision, validation, visualization, writing - original draft. Lizhi Liu: Formal analysis, project administration. Tongwei Lu: Funding acquisition, writing - review and editing. Acknowledgements This work was funded by the National Natural Science Foundation of China, Grant Number 62171328 and funded by Construction of Intelligent Robot Platform, Grant Number 2018ZYYD059. Conflict of interest The authors have declared no conflict of interest. Open Research Data availability statement Research data are not shared. References 1Girshick, R., et al.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 580– 587. IEEE, Piscataway (2014) 2Ren, S., et al.: Faster r-cnn: Towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, vol. 28. MIT Press, Cambridge (2015) 3Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440– 1448. IEEE, Piscataway (2015) 4He, K., et al.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2961– 2969. IEEE, Piscataway (2017) 5Redmon, J., et al.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779– 788. IEEE, Piscataway (2016) 6Redmon, J., Farhadi, A.: Yolo9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7263– 7271. IEEE, Piscataway (2017) 7Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. arXiv preprint arXiv:180402767 (2018) 8Bochkovskiy, A., Wang, C.Y., Liao, H.Y.M.: Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:200410934 (2020) 9Liu, W., et al.: Ssd: Single shot multibox detector. In: European Conference on Computer Vision, pp. 21– 37. Springer, Berlin (2016) 10Fu, C.Y., et al.: Dssd: Deconvolutional single shot detector. arXiv preprint arXiv:170106659 (2017) 11Zhou, X., Wang, D., Krähenbühl, P.: Objects as points. arXiv preprint arXiv:190407850 (2019) 12Law, H., Deng, J.: Cornernet: Detecting objects as paired keypoints. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 734– 750. Springer, Berlin (2018) 13Tian, Z., et al.: Fcos: Fully convolutional one-stage object detection. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9627– 9636. IEEE, Piscataway (2019) 14Carion, N., et al.: End-to-end object detection with transformers. In: European Conference on Computer Vision, pp. 213– 229. Springer, Boston (2020) 15Lin, T.Y., et al.: Microsoft coco: Common objects in context. In: European Conference on Computer Vision, pp. 740– 755. Springer, Boston (2014) 16Lin, T.Y., et al.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117– 2125. IEEE, Piscataway (2017) 17Hong, M., et al.: Sspnet: Scale selection pyramid network for tiny person detection from uav images. IEEE Geosci. Remote Sens. Lett. 19, 1– 5 (2021) 18Deng, C., et al.: Extended feature pyramid network for small object detection. IEEE Trans. Multimedia. 24, 1968– 1979 (2021) 19Gong, Y., et al.: Effective fusion factor in fpn for tiny object detection. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1160– 1168. IEEE, Piscataway (2021) 20He, K., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770– 778. IEEE, Piscataway (2016) 21Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556 (2014) 22Lin, T.Y., et al.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980– 2988. IEEE, Piscataway (2017) 23Jocher, G., et al.: Ultralytics/yolov5: v5. 0-yolov5-p6 1280 models aws supervise. ly and youtube integrations. Zenodo, Geneva, Switzerland (2021) Volume59, Issue3February 2023e212732 FiguresReferencesRelatedInformation