SD-YOLO: An Attention Mechanism Guided YOLO Network for Ship Detection

Authors: yunze  zhang
ying  hao
yifei  li
Venue: IEEE_org_ICIST_2024_Conference
Published: 2024-08-15 03:57:13

Abstract: Synthetic aperture radar (SAR) has become increasingly vital in ship detection, owing to the significance of highresolution images it provides. Ship detection from SAR images is a huge challenge, and the main difficulty is the small size and low resolution due to the long observation distance, resulting in high false negative rate. To solve this problem, a new detection model is proposed in this paper called Ship Detection YOLO (SD-YOLO). It improves the detection accuracy of small targets with almost no sacrifice of the algorithm’s real-time performance. Specifically, we improve the C3 module in YOLOv5 called CB-C3 by combining coordinate attention (CA) and Bottleneck. Moreover, in order to improve the ship detection precision and training efficiency, the α-IoU loss function constraint detection bounding box is employed, so that the new detector can more efficiently acquire the position of the ship object. Then, we redesign the neck layer of YOLOv5 with bidirectional feature pyramid network (BiFPN), which can better integrate multi-scale features. Finally, experimental results on multiple public SAR datasets show that the average precision (AP) of SD-YOLO can reach 96.1% in SAR ship detection dataset (SSDD) and 73.2% in large-scale SAR ship detection dataset (LS-SSDD), which is an increase of 2.7% and 7.9% respectively over the previous improvement. Compared with most mainstream algorithms, SD-YOLO has fewer parameters, only 6.79M.

Main paper: 
 i. introduction: 
 Ship detection is crucial for a wide range of applications, including search and rescue operations, maritime traffic control, and automated fisheries management [1] . Although numerous ship detection methods exist, the marine environment presents unique challenges, such as varying weather conditions, ocean waves, and other unpredictable natural factors, which complicate detection efforts. In this context, Synthetic Aperture Radar (SAR) offers a significant advantage due to its ability to capture high-resolution images over large areas, regardless of weather or lighting conditions. SAR is particularly effective in detecting vessels on the ocean surface [2] , making accurate and real-time SAR-based ship detection results essential.Traditional ship detection models rely on manual feature extraction and prior knowledge, making it challenging to effectively capture target characteristics. Consequently, these models often lack robustness and generalization capabilities. In contrast, deep learning has made significant advancements in computer vision, particularly in areas such as semantic segmentation [3] , object recognition [4] , and object detection [5] . By learning key features from large-scale datasets, deep learning algorithms have demonstrated superior accuracy in object detection and exhibit strong robustness.Deep learning-based detection methods are typically classified into two categories: two-stage and single-stage algorithms [6] . Two-stage detectors include models such as Region-based Convolutional Neural Networks (R-CNN) [7] , Fast R-CNN [8] , and Faster R-CNN [9] . R-CNN generates region proposals and normalizes them before performing classification. Fast R-CNN and Faster R-CNN compute features at a single scale, striking a balance between accuracy and processing efficiency. However, these models are generally too slow to meet the realtime processing requirements of embedded systems [10] . To address the demand for faster detection, single-stage models have been developed. These detectors, such as Corner-Net, You Only Look Once (YOLO) [6] , and Single Shot MultiBox Detector (SSD) [11] , bypass the region proposal stage and directly perform detection on densely sampled positions. YOLO, in particular, utilizes a feed-forward convolutional network for both object localization and classification [12] , making it a popular choice for ship detection tasks.Despite its strengths, YOLO does not effectively account for the spatial relationships between objects in input images [13] , which hampers its detection performance, particularly for small vessels. To address this issue, various studies have proposed improvements to the YOLOv5 model in recent years. For instance, Zheng et al. [14] improved the loss function by introducing Distance Intersection over Union (DIoU) and accounting for the Euclidean distance between actual center points, leading to better detection accuracy. Likewise, Sun et al. [15] used a combination of top-down and bottom- up pathways to achieve more thorough feature fusion. By linearizing the equations and accurately identifying vessel position coordinates, they further increased the model's precision. While the aforementioned algorithms improve feature extraction network architectures, they fail to effectively integrate contextual information in the final model layers, which is crucial for accurate predictions [14] , [15] .Moreover, training deep learning models for target detection requires large datasets. The primary datasets used for synthetic aperture radar (SAR) ship detection are the SAR Ship Detection Dataset (SSDD) [16] and the Large-Scale SAR Ship Detection Dataset (LS-SSDD) [17] . Key differences exist between SAR and natural images: SAR images are singlechannel, sparsely distributed, and contain relatively few ships. As a result, when generic object detection models are applied directly to SAR ship detection tasks, their performance tends to degrade [18] . This underscores the urgent need for a highly accurate and robust SAR-specific ship detection model.In this study, considering the unique characteristics of SAR images, we leverage the speed of the YOLOv5 algorithm to improve ship detection accuracy. The key contributions of this work are outlined as follows:1) Inspired by the coordinate attention mechanism, we developed a novel feature enhancement module called CB-C3. Unlike previous approaches [19] , CB-C3 emphasizes foreground information in input images, significantly improving the model's detection performance, particularly for small vessels.2) The neck layer integrates a Bidirectional Feature Pyramid Network (BiFPN), which is more efficient than YOLOv5 [12] at extracting and fusing multi-scale feature information, leading to improved accuracy in ship detection.3) The loss function is enhanced with α-IoU, which extends the traditional IoU loss into a power IoU loss by adjusting the α parameter. Unlike previous methods [20] , [21] , α-IoU allows the detector to achieve multi-level bounding box regression when training on different SAR ship datasets.YOLO is one of the most advanced real-time object detection systems available today and has been widely adopted in ship detection [6] . Unlike the R-CNN series [5] , YOLO approaches object detection as a regression problem. The YOLO detection process is depicted in Figure 1 . To generate a fixed-size output after applying convolutions to the entire image, the input is resized to 416 × 416 pixels. This resized image is then divided into a grid of cells, each tasked with detecting potential bounding boxes. Each cell provides five parameters: w, h, x, y and conf idence [6] . Here, (w, h) represent the dimensions of the target's bounding box, while (x, y) denote its coordinates. The conf idence score reflects both the likelihood of an object being present within the cell and the accuracy of the predicted bounding box. In cases where multiple bounding boxes detect the same object, YOLO uses Non-Maximum Suppression (NMS) to select the most accurate one.The input layer serves as the entry point for the object detection model, where adaptive image resizing is applied to improve generalization across various datasets. After preprocessing, the images are processed by a convolutional neural network (CNN). The backbone network, which forms the core of the model, is built upon architectures like AlexNet, ResNet, VGGNet, and GoogLeNet [12] . YOLOv5s introduces several architectural enhancements over YOLOv4, including the addition of the Focus module in its backbone, which is not present in YOLOv4. Furthermore, YOLOv5s incorporates two distinct versions of the Cross Stage Partial Network (CSPNet), distinguishing it further from its predecessor. In the neck layer, YOLOv5s utilizes the Path Aggregation Network (PANet) alongside CSPNet to enhance feature fusion [10] . Finally, in the head module, YOLOv5s replaces DIoU-NMS with Weighted NMS, resulting in a slight performance improvement without increasing computational complexity.The use of remote sensing technology for ship detection has been a critical area of research in both military and civilian applications [22] . However, Synthetic Aperture Radar (SAR)based ship detection continues to face challenges, including strong target scattering, multi-scale variations, and background interference, which reduce detection accuracy. To address these issues, innovative solutions like NAS-YOLOX have emerged, combining Neural Architecture Search with a multiscale attention mechanism to enhance feature extraction and fusion, thereby improving detection precision [22] . In addition to SAR-based methods, researchers have explored alternative approaches to enhance object detection performance in remote sensing images (RSI). For instance, CamoNet, an object camouflage network, introduces imperceptible perturbations to deceive CNN-based detectors, improving detection capabilities [23] . Moreover, efforts have been made to integrate low-precision floating-point algorithms with quantized neural networks, enabling more efficient processing of deep learning models for ship detection, such as in satellite image-based semantic segmentation tasks [24] . Further research has focused on developing custom convolutional neural networks, specifically training SAR ship detectors from scratch to meet unique requirements and enhance detection accuracy [25] .To address class imbalance in SAR ship detection datasets, interpretable evidence learning has been proposed as a solution for learning from biased samples [26] . High-resolution feature generators have also been developed to improve the detection of small ships in optical remote sensing images, tackling ongoing challenges in this domain [27] . Optimized models, such as YOLO-OSD, leverage hybrid data-driven approaches to enhance ship localization in multi-resolution SAR images [28] . Additionally, unsupervised domain adaptation techniques based on cross-domain feature interaction and balanced data contribution have been introduced to improve detection performance across diverse datasets [29] . Collectively, these advancements underscore a range of methods leveraging remote sensing technology to enhance ship detection accuracy.The SD-YOLO framework, illustrated in Figure 2 , incorporates CSPDarknet53 as its backbone, similar to YOLOv5, to extract multi-scale features from SAR images. To effectively capture contextual information, the framework introduces a novel CB-C3 module during the feature extraction stage. For multi-scale feature fusion, the high-performance BiFPN is employed in the neck layer. Detailed descriptions of these components are provided in Sections B and C. Given the importance of bounding box regression in determining the model's detection accuracy, we introduce the α-IoU, a power IoU loss family, during the prediction and classification stages. The α-IoU generalizes the loss function through a unified power parameter. A comprehensive explanation of this loss function is provided in Section D.Attention mechanisms are commonly used in object detection tasks to help models focus on key areas, particularly the foreground of the image. Since the introduction of attention techniques like squeeze-and-excitation networks [30] , efficient channel attention [31] , and the convolutional block attention module [32] , the detection performance of YOLO models has seen significant improvement. However, the increased computational overhead associated with these mechanisms has limited their use, particularly in networks with lower computational capacity. To address this issue, we propose the Coordinate Attention (CA) mechanism, designed to enhance the network's accuracy without adding substantial computational burden. The Coordinate Attention (CA) mechanism is designed for simplicity and adaptability, making it suitable for use across various neural network architectures. A comprehensive discussion of CA can be found in [33] , which highlights its superior performance compared to traditional attention mechanisms. CA improves the efficiency of information transmission by guiding convolutional neural networks to concentrate on important coordinates while filtering out irrelevant information.The C3 module in YOLOv5 offers fewer parameters and faster inference speed; however, its simple convolutional layer structure leads to poor performance in detecting small objects. To overcome this limitation, we propose the CB-C3 module, with its detailed structure shown in Figure 3 . In the original C3 module, the bottleneck structure primarily relies on 3 × 3 convolutions for feature extraction, which limits the receptive field. To address this, we integrated average pooling within the CA mechanism to enhance YOLOv5's ability to capture global information more effectively.In the YOLOv5 architecture, the Concat operation serves as the feature fusion module in the neck layer, concatenating input tensors along a specified dimension to enable higherdimensional data processing. While this method effectively merges multiple tensors, it has drawbacks, such as increased computational load and the risk of dimensionality explosion. To improve the multi-scale fusion of ship features extracted from the backbone, we redesigned YOLOv5's neck layer by incorporating the BiFPN.Feature fusion networks are crucial for capturing the contextual significance of input images. The BiFPN, introduced by Mingxing Tan in July 2020, offers two key advantages [18] . First, it enhances the efficiency of feature fusion compared to the original FPN by integrating a downsampling path that transitions from high to low resolutions. Unlike PANet, BiFPN is both more efficient and lightweight, as it removes nodes that rely solely on single inputs [34] . Second, BiFPN introduces skip connections between inputs and outputs, enabling the network to more effectively combine low-level and high-level features. Stacking multiple BiFPN modules allows the model to progressively refine feature fusion as its depth increases. The structure of the BiFPN module is illustrated in Figure 4 , where X i represents the feature from the i-th input.Intersection over Union (IoU) is a critical metric for evaluating object detection algorithms and calculating Average Precision (AP). IoU measures the ratio of the overlapping area between the predicted bounding box and the ground truth box to their union. However, when there is no overlap between the predicted and actual boxes, the gradient becomes zero, rendering the traditional IoU calculation ineffective. To address this issue, improved IoU methods, such as Generalized IoU (GIoU) [5] , have been developed. GIoU adds a penalty term to the loss function, allowing the model to continue learning even in cases where the predicted and ground truth boxes do not overlap. Further advancements, including DIoU [35] and Complete IoU (CIoU) [21] , incorporate additional factors such as aspect ratio. These methods are designed to handle situations where one bounding box fully overlaps another, further reducing the impact of the gradient vanishing problem.However, there are large differences between SAR images and optical images, for example, the ship targets in SAR images are smaller and more widely distributed, and the above algorithms have certain limitations when detecting ships in SAR images. In order to improve the prosperity of the algorithm, this paper employs a new loss function called α-IoU [36] . The general IoU loss is defined as L IoU = 1 -IoU. The α-IoU first use the Box-Cox transformation and generalize the IoU loss to an α-IoU loss:EQUATIONEQUATIONfurthermore, the YOLOv5 IoU loss function can be generalized as:EQUATIONwhere b, b gt denote the central of anchor box and target box respectively. ρ, defined as ∥b -b gt ∥ 2 , quantifies the Euclidean distance. C denotes the smallest bounding region that includes both the ground truth box and the predicted box. By adjusting the α parameter, the current IoU-based loss can be extended into a new family of IoU losses. This new family comprises the IoU term and an additional regularization term governed by α.To assess the performance of our proposed method, we employed the SSDD [16] and LS-SSDD [17] datasets. SSDD [16] is an essential publicly available dataset for SAR ship detection, containing 1,160 SAR images and 2,456 ship instances. LS-SSDD [17] , introduced by Li et al. in 2020, addresses the issue of limited data volume in SSDD [16] Table I . And we converted both datasets into YOLO and COCO formats.The network is trained on a Linux system equipped with an NVIDIA GTX3090 GPU with 24GB of memory and an Intel(R) Core(TM) i9-10940X CPU. Various maritime target detection algorithms were compared and evaluated based on The formulas for calculating Precision, Recall, and IoU are as follows with True Positive (TP), False Positive (FP) and False Negative (FN):EQUATIONEQUATIONEQUATIONwhere S over refers to the overlapping area between the ship target bounding box and the ground truth, while S union denotes the area of their union. An IoU value exceeding 0.5 means that the predicted bounding box overlaps with at least half of the ground truth, which qualifies as a correct object detection and corresponds to AP 50 . The definitions for AP 75 and AP 50:95 are based on a similar concept.To demonstrate the effect of the newly proposed loss function and innovative network design, ablation studies were performed on both the SSDD [16] and LS-SSDD [17] datasets. For consistency and fairness, the original YOLOv5 method served as the baseline for comparison. Initially, during the optimization of the α-loss function, experiments were performed to determine the optimal α values for different datasets, with the goal of maximizing the algorithm's accuracy through gradient descent. Table II presents the experimental results for various α values.We observed that for LS-SSDD [16] , the highest accuracy is achieved when α=3. In contrast, for SSDD [17] , the optimization is more effective when α=5. Consequently, we select the most suitable α value for each detection scenario. To verify the effectiveness of optimizing YOLOv5, we combined three different optimization methods and conducted ablation experiments on the SAR datasets. It is evident that the algorithm's precision improves, with AP 50 increasing by 1.3% for SSDD [16] and 6% for LS-SSDD [17] . Furthermore, when YOLOv5 is further optimized using BiFPN and the new loss function, the algorithm's performance is enhanced, achieving AP 50 values of 96.1% and 73.2%, respectively.To assess the overall effectiveness of the proposed SD-YOLO, we compared it with several recent advanced methods. These methods include Cascade R-CNN [37] , YOLOX [19] , Faster R-CNN [9] , Tridentnet [38] , and YOLOv4 [20] . Ensuring fairness, all algorithms were tested using the same batch size. The results validate the effectiveness and accuracy of the image processing method, network design, and loss function optimization.Table IV presents the numerical outcomes obtained by various methods on SSDD [16] and LS-SSDD [17] datasets. The experimental findings across multiple scenarios indicate that this method achieves superior accuracy compared to other existing approaches. During SSDD [16] training, SD-YOLO's AP 50 exceeds Centripetalnet [39] by 5.7%, and in LS-SSDD [17] , it outperforms Sparse R-CNN [40] by 19%. Moreover, our method ranks just behind YOLO-X [19] in FPS performance. Despite this, the approach discussed in this paper maintains a strong advantage in real-time execution. The experimental outcomes under typical scene conditions are illustrated in Fig. 5 . For smaller targets, YOLOv5 experiences some missed detections, whereas SD-YOLO shows impressive robustness and a high recognition rate.Based on the experimental data in Table II , we observed that as α increases, the model's accuracy improves to a certain extent. However, the α value should not be increased indiscriminately to enhance detection performance. For LS-SSDD [17] , when α exceeds 3, the algorithm's accuracy tends to decrease. We believe that indiscriminately increasing the α value may lead to overfitting issues. Therefore, different α values should be selected for various application scenarios. By analyzing the first and second sets of data in Table III , we discovered an interesting phenomenon: when the CA structure is added to C3, the number of parameters in the algorithm does not increase but actually decreases to some extent. We propose two possible reasons for this. First, compared to traditional parameter matrices, CA introduces a parametersharing mechanism that reduces the number of parameters at each position by learning attention from coordinate positions. Second, CA incorporates dimensionality reduction operations, which effectively represent coordinate information and reduce the dimensionality of input features, thereby decreasing the number of parameters. In the comparison experiments, although the YOLOv5 algorithm has fewer parameters than YOLO-X, YOLO-X achieves a higher FPS. We attribute this to the structural differences between the algorithms.In this paper, we present a novel ship detection model that improves detection accuracy with minimal increases in algorithmic complexity. Unlike traditional ship detection methods, our approach incorporates an attention mechanism based on the CB-C3 module, which adjusts the weight distribution of features across different scales using 2D global pooling. To further enhance the model's sensitivity to ship targets, we optimize YOLOv5's neck layer by integrating the BiFPN. Additionally, we introduce the α loss function in SD-YOLO to refine the final classification, taking into account the unique characteristics of ships in remote sensing scenarios. This ensures that the classification process is fast, stable, and precise. Experimental results demonstrate that this architecture outperforms benchmark methods in terms of robustness and accuracy on publicly available SSDD and LS-SSDD datasets.Key challenges in SAR ship detection include dense target distribution, significant dynamic variations, and scene element occlusion. Our future research will focus on developing robust techniques for detecting densely packed targets at multiple scales.  
 
 Appendix:  



Reviews: review: title: title: Manuscript Accept
main_review: main_review: This work is novel. Some issues should be considered in the revision.
Can CB-C3 improve the detection performance of the other models except the small ships?
How does the α-IoU loss function constraint detection bounding box improve the detector performance?
What is the main challenge of having a detection in a large-scale SAR ship detection dataset?
title: title: The paper is logically clear, the simulation results are credible, and it is recommended for publication.
main_review: main_review: This paper proposes SD-YOLO, an enhanced detection model for ship detection in SAR images, which improves detection accuracy for small targets through modified C3 module, α-IoU loss, and BiFPN neck layer, achieving state-of-the-art performance with reduced parameters. The paper is logically clear, the simulation results are credible, and it is recommended for publication. This paper is well organized and contains meaningful results. However, the following comments should be considered in the revision.
1.The SD-YOLO model has shown promising results in ship detection from SAR images. To further demonstrate its versatility, future research could explore extending the model to detect other maritime objects such as buoys, oil spills, or even marine life. This would not only validate the generalizability of the model but also provide valuable tools for a wider range of maritime surveillance and monitoring applications.
2.Although the SD-YOLO model achieves high accuracy on benchmark datasets, deploying it in real-world scenarios where large amounts of labeled data may not be readily available can be challenging. Incorporating transfer learning techniques by pre-training the model on a larger, more diverse dataset and then fine-tuning it on the target dataset with limited annotations could help overcome this issue. This approach would make the model more accessible and applicable in practical applications.
3.The current SD-YOLO model focuses on detecting ships in individual SAR images. However, with the availability of video SAR data, incorporating spatio-temporal information across frames could potentially further improve detection accuracy. For instance, utilizing optical flow or tracking algorithms to model the motion of ships between frames could help reduce false positives and increase robustness to occlusion or noise in individual frames. Additionally, this could enable the model to predict future ship positions, providing valuable insights for maritime traffic management and collision avoidance systems.
title: title: A new ship detection model is proposed in this paper. This paper is interesting. However, the following comments should be considered in the revision.
main_review: main_review: Question 1:
Please provide further details on how the CB-C3 module specifically enhances the detection of small ships compared to previous methods. How does the increased focus on foreground information translate into improved performance metrics?
Question 2:
Please elaborate on how the BiFPN neck layer improves the extraction and fusion of multi-scale features? What specific advantages does it offer over YOLOv5 in terms of ship object detection accuracy?
Question 3:
How does the α-IoU loss function's adjustment of the parameter α lead to more effective bounding box regression across multiple SAR ship datasets? Can the authors discuss the benefits of this approach compared to existing IoU loss functions and provide examples of its impact on detection performance?