# The Big Picture: An Improved Method for Mapping Shipping Activities
## 4 Improving AIS Data
### Vessel Detection Based on Satellite Images  
Several steps are required for extracting detections of vessels from satellite imagery. In this work, we present a pipeline suitable for both optical and SAR images, as provided by the European initiative Copernicus ([[https://www.copernicus.eu/en](https://www.copernicus.eu/en)]([https://www.copernicus.eu/en](https://www.copernicus.eu/en)), (accessed on 6 September 2023)); with data coming from Sentinel-2 and Sentinel-1, respectively [58; 59]. Since optical images are very different from SAR images, different preprocessing steps for each kind of imagery are required. For SAR imagery (i.e., Sentinel-1 data), we perform corrections, such as converting SAR geometries to geo-referenced geometries using GDAL tools in Python (e.g., gdalwarp). SAR images come as single band images in three different resolutions: 10 m, 20 m and 60 m. The optical images (i.e., Sentinel-2 data) come in RGB bands together with an infrared (IR) band. We stack the RGB and the SWIR bands together and we compose a panchromatic image in the GeoTIFF format, using 16-bit encoding, and we perform pan-sharpening to increase its resolution. The panchromatic image is created by computing the average of all the 10 m resolution bands, that is, RGB and IR in order to obtain a single high-resolution image. The aim of the pan-sharpening technique is to fuse the higher spatial information from the panchromatic image and the spectral information from a lower spatial information multi-spectral image. A true color image (TCI) in 10 m is readily available in a Sentinel-2 image directory; however, we prefer to compose the panchromatic pan-sharpened image using the individual bands since the TCI image comes in 8-bit encoding, which is lossy. Maintaining a 16-bit encoding will be crucial for the filter stage which follows. After we have preprocessed the images, we divide them into smaller tiles, with dimensions of 256 \(\times\) 256 pixels, in order to be further processed more easily. Each pixel is of 10m length in each side, meaning that it covers an area of 100 m\({}^{2}\). Since we intend to use an AI-based approach for detecting vessels in satellite imagery, feeding thousands of satellite images which correspond to several Terabytes of data into the network can create a serious bottleneck; it will compromise the performance and applicability of our approach in a density maps use case, in which thousands of satellite images need to be processed. Thus, we filter out the image tiles which are redundant, meaning the ones for which we have indications that they do not contain vessels before we feed them to the neural network. For example, a Sentinel-1 tile that is totally black depicts the sea and no object appears in it. For Sentinel-1 image tiles, we use statistics and thresholding (i.e., amount of black/white pixels), while for Sentinel-2 image tiles, we use thresholds that are based on the difference of pixel values between the red band and the infrared band (R-SWIR). We also use the ACL as a mask to filter out clouds. Eventually, the filtering step results in image tiles sizing up to at least one order of magnitude less than the size of the original image. For the vessel identification in the preprocessed satellite images, the YOLOv4 neural network architecture [60] was selected. The YOLO network is a CNN-based state-of-the-art solution that has proven to be very efficient in similar tasks of determining moving objects in images (ranging from moving cars to aircrafts) [61; 62; 63; 64; 65]. Providing high accuracy with low response time, the YOLOv4 is highly suitable for handling large volumes of data in an effective manner. In recent years, the effectiveness of the YOLOv4 version has been evaluated for detecting vessels in SAR and optical images [66; 67]. We train the NN with vessels detected in satellite images, using AIS data as ground truth. Since we also want to detect the precise location of vessels, we used an object detection framework instead of a simple CNN.