# Probabilistic Ship Detection and Classification Using Deep Learning
## 1 Introduction  
Accurate detection and reliable classification of nearby moving ships are essential functions of an autonomous ship, being closely linked to safe navigation [1; 2]. When a ship navigates, the chance of collision with other ships is possible in various directions, such as those that overtake, approach head-on, or cross the autonomous ship. The International Regulations for Preventing Collisions at Sea (COLREGs) defines several rules to prevent collisions [3]. In particular, overtaking (rule 13), head-on (rule 14), and crossing (rule 15) situations are considered potential collision scenarios. Autonomous ships mainly collect information related to moving obstacles through non-visual sensors such as radar [4] and automatic identification systems (AIS) [5]. However, recognizing the nearby ships reliably is difficult if only using information collected from non-visual sensors to determine whether these are dangerous obstacles. Therefore, autonomous ships must recognize dangerous obstacles using a visual camera. This problem is similar to the detection of cars, pedestrians, lane, or traffic signs using a camera in autonomous vehicles. Hitherto, some research concerning ship detection and classification has been reported. For example, seashore ship surveillance and ship detection from spaceborne optical images have been achieved [6; 7; 8; 9]. Synthetic aperture radar (SAR) imagery was used to detect ships and objects on the surface of the earth [7; 8]. [PERSON] et al. used artificial neural networks (ANN) for ship detectionwith X-band Kompsat-5 SAR imagery [9]. Unfortunately, most of the existing research focused only on ship detection based on spaceborne optical images, such as SAR imagery. Furthermore, these studies focused on visual ship detection based only on a single image. All previous works on ship detection and classification were based on a still image. To the best of our knowledge, no studies exist for the detection of ships using an image sequence or a video. In this study, we propose a novel probabilistic ship detection and classification method using deep learning. This method considers the confidence from a deep learning detector as a probability and the probabilities from consecutive images are combined over time via Bayesian fusion. To the best of our knowledge, no research work has used the confidence from a deep learning detector in a Bayesian framework. The proposed ship detection system involves three steps. In the first step, ships are detected for each frame using Faster R-CNN [10]. In the second step, the detected ships are gathered over time and the missed ships are recovered using the Intersection over Union (IoU) of the bounding boxes between consecutive frames. The corresponding detection confidence is updated and the recovery compensates for the misdetection confidence over a few frames. This approach ensures robust ship detection and minimizes misdetection. In the third step, the probabilities from the Faster R-CNN are combined over time and the classes of the ships are determined by Bayesian fusion. The use of Bayesian fusion was supported by its reported use in prior studies [11; 12]. To use a deep learning framework in ship detection, a ship dataset was needed to train the Faster R-CNN. Well-known image datasets, such as ImageNet [13], PASCAL visual object classes (VOC) challenge [14], and Microsoft common objects in context (MS COCO) [15], include ship images but the number of ship images is limited and the various classes of ships are not labeled. Popular intelligent transportation system (ITS) datasets, such as the Karlsruhe Institute of Technology and Toyota Technological Institute (KITTI) dataset [16], also does not include ship images. Because no public dataset exists for ship detection in the sea environment, we manually collected thousands of ship images from Google image search and built our own ship dataset. The contributions of this paper can be summarized as follows.