# Integrating Prior Knowledge into Attention for Ship Detection in SAR Images
###### Abstract  
Although they have achieved great success in optical images, deep convolutional neural networks underperform for ship detection in SAR images because of the lack of color and textual features. In this paper, we propose our framework which integrates prior knowledge into neural networks by means of the attention mechanism. Because the background of ships is mostly water surface or coast, we use clustering algorithms to generate the prior knowledge map from brightness and density features. The prior knowledge map is later resized and fused with convolutional feature maps by the attention mechanism. Our experiments demonstrate that our framework is able to improve various one-stage and two-stage object detection algorithms (Faster R-CNN, RetinaNet, SSD, and YOLOv4) on two benchmark datasets (SSDD, LS-SSDD, and HRSID).  
prior knowledge; expert knowledge; SAR images; object detection; attention mechanism +
Footnote †: journal: Medical  
before using CNNs to further detect ship targets. Their method substantially improved the detection capability of offshore ships while ensuring the inshorse ship detection performance. However, due to the over-suppression of saliency detection, their method did not perform well when facing small ships with very few pixels. [PERSON] et al. [31] pointed out that the general deep learning-based detectors perform poorly because they ignore the characteristics of SAR images. They grouped all the extracted strong scattering points, and generated the initial ROI areas based on the grouping result. Compared to those extracting only a portion of strong scattering points as key points, [PERSON] et al.'s method performed better for small targets. However, the method of getting the initial ROI at the beginning of training was trained insufficiently, which makes the initial ROIs inaccurate and causes the model to fail to learn valid knowledge. Although previous works recognize the importance of prior knowledge for the CNNS, they combine the prior knowledge by feeding it into the networks. Therefore, the networks could not access the raw information which may be missing from the prior knowledge.  
In this paper, we propose a framework which integrates prior knowledge into CNNs by fusing it with an attention mechanism. In our model, the prior knowledge is used to guide the attention of deep networks. Our framework can be categorized into two stages: prior knowledge representation and prior knowledge integration. The aim of the first stage is to generate the probability score for each pixel of which how likely it is on the ship area or background. Based on the findings from other works [28; 31; 32], brightness and density are two promising features for prior knowledge. To this end, we employ k-means clustering as brightness filtering because there are ships and backgrounds in a bounding box. Similarly, DBSCAN clustering is utilized as the density filtering. For the second stage, we down-pool the original prior knowledge map into the same size as the feature map. After that, position-wise multiplication integrates the resized prior knowledge map into the CNNs.  
Our contributions can be summarized into three items.  
1. To our best knowledge, our framework is the first that can integrate prior knowledge into arbitrary CNN-based detectors using attention mechanism for SAR images.
2. By the attention mechanism, the deep learning models can learn from both prior knowledge and vanilla images.
3. Our experiments exhibit the superiority of our method using object detection algorithms (Faster R-CNN, RetinaNet, YOLOv4, and SSD) on three SAR image datasets (SSDD, LS-SSDD, and HRSID).  
The remainder of this paper is organized as follows. Section 2 presents the related works about traditional methods, CNN-based object detection algorithms, and attention mechanisms. Section 3 details our intuition and approaches to represent prior knowledge. Section 4 demonstrates the combination of prior knowledge and deep CNNs. Section 5 displays the settings, descriptions, and results of our experiments.