Optical Target Recognition for Drone Ships

Remote controlled drone ships without crews on board are expected by the end of the decade. To achieve the goal of developing (semi-)autonomous boats, reliable vision-based methods for vessel detection, classification, and tracking are needed. In this paper, we present a machine learning approach for vessels detection from a moving and zooming camera. In particular, the proposed method is supervised and derives from a fast and robust people detection algorithm. Quantitative experimental results have been obtained on a publicly available data set, which contains images from real sites, demonstrating the effectiveness of the approach. Ground truth annotations and the code of the proposed algorithm are both released for the community. raindrops) contribute to generate a highly dynamic scenario (Bloisi et al., 2014). 2.1 Vessel Detection and Tracking Vessel detection in the maritime scenario requires the monitoring of large areas at different resolution levels. Indeed, the objects of interest can have very different size, ranging from few to hundreds of meters in length. SeeCoast system (Rhodes et al., 2006) detects, classifies, and tracks vessels by fusing electro-optical (EO) and infrared (IR) video data with radar and AIS data. The detection is carried out by estimating the motion of the background and segmenting it into components. However, motionbased vessel detection can experience difficulties when a boat is moving directly toward the camera or is anchored off the coast due to the small amount of inter-frame changes. Maximum Average Correlation Height (MACH) filters are employed for vessel classification by Sullivan & Shah (2008). Vessel detections are crossreferenced with ship pre-arrival notices in order to verify the access of vessels to the port. As reported by the authors, such an approach tends to misclassify small boats. Fefilatyev et al. (2010) propose a system exploiting a non-stationary camera installed on an untethered buoy. After detecting the horizon line, a color gradient filter is applied to obtain a grayscale image with intensities corresponding to the magnitude of color changes, then thresholding on the grayscale image is used to find objects of interest. The algorithm is limited by the assumption that all marine targets are located above the horizon line. ASV (Pires et al., 2010) is an automatic optical system for maritime safety using IR, GPS, and AIS. To detect relevant objects, the sea area is segmented and its statistical distribution is calculated. Any irregularities from this distribution are supposed to correspond to objects of interest. However, due to wakes, such an approach can produce false positives. An object detection system for finding ships in maritime video based on Histogram of Oriented Gradients (HOG) is described by Wijnhoven et al. (2010). Since the calculation of the detection features involves a significant amount of computational resources, realtime performance can be obtained only by means of hardware acceleration with programmable components such as FPGAs. 2.2 Floating Objects Detection Floating objects detection plays an important role in USVs (Unmanned Surface Vehicles). Indeed, obstacles in the operational environment can be floating pieces of wood or other debris, which presents a significant challenge to continuous detection from images taken on-board (Kristan et al., 2016). Snyder et al. (2004) state that transitory obstacles, such as floating debris, are best detected and analyzed at navigation time with visual means. They propose a system for fully autonomous navigation in a river scenario. Obstacles and objects of interest are tracked across multiple cameras (by using feature clusters in aspect-elevation space) and then mapped onto the 3D world. Distant moving objects are detected and tracked by clustering feature points, while nearby movers are detected with motion blobs. Stereo rigs are used by Huntsberger et al. (2011) for obstacle and moving objects detection on a USV. However, since a large baseline is required for granting a large field of view coverage, this can create instability for small vessels. A method for detecting water regions in videos by clustering color and texture features is proposed by Santana et al. (2012). Fefilatyev et al. (2009) use the horizon line position to eliminate all edges not belonging to floating objects: it is assumed that within an image, all objects of interest lie above the horizon line. A similar idea is exploited by Wang et al. (2011): They first detect the horizon line and then search for a potential obstacle in the region below the horizon. The main drawback of approaches based on the horizon line detection is that situations in coastal waters, close to the shoreline, cannot be easily handled, since the edge of water does not correspond to the horizon. 2.3 Humans Detection In many USV applications, humans are considered just a special case of obstacles to be avoided (Almeida et al., 2009). However, distinguishing between human and non-human detections is important in order to devise different and more opportune navigation strategies, for example in case of casualty detection for search and rescue operations. Differently from a static floating objects, a person may be swimming or diving, possibly without being aware of an approaching USV, and this poses serious concerns for human safety. In recent years, the use of thermal images from Forward-Looking Infrared (FLIR) cameras for target detection and tracking has become popular in various application domains (Sanna & Lamberti, 2014). Independent FLIR cameras have also been mounted on vessels and used for target detection and tracking at sea (Kim & Lee, 2014). Martins et al. (2013) uses a combination of thermal and colour images to detect casualties from a USV in the context of the FP7 project ICARUS. A simple horizon detection algorithm was applied to the thermal image to limit the search space to the water surface only, then targets detected by both (fixed and calibrated) cameras were tracked using Kalman filtering to deal with false positives and temporary false negatives. Although computationally efficient, the detection coverage of the proposed solution is limited by the narrow field of view and by the minimum focusing distance of the thermal camera, which made the system unsuitable for tracking humans in proximity of the USV. To increase the water surface area covered by the cameras, the latter can be mounted on a stabilized pan-tilt unit (PTU), as proposed by Bibby & Reid (2005). Underwater human detection can be achieved instead by using multibeam or mechanically-rotated sonars, similarly to what is already done for obstacle detection with autonomous surface vehicles (Heidarsson & Sukhatme, 2011).