# The Big Picture: An Improved Method for Mapping Shipping Activities
## 4 Improving AIS Data
### Vessel Detection Based on Satellite Images
The object detection framework deploys a CNN per image pixel and determines the bounding box of the detected object. Our trained model is able to detect vessels with a precision of 92%, as shown in the experiment results provided in Table 1.  
For each image, the output of the vessel detection includes (a) the bounding box of the detected vessel (with respect to the centre of the image), (b) the class of the detected object (e.g., vessel, tanker, etc. ), and (c) a confidence value that indicates the possibility of \begin{table}
\begin{tabular}{c c} \hline \hline
**Metric** & **Value** \\ \hline Precision & 92\% \\ Recall & 93\% \\ F1-score & 92\% \\ \hline True Positive (TP) & 80\% \\ False Positive (FP) & 7\% \\ False Negative (FN) & 6\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Vessel detection experiment results using 400 semi-manually labelled Sentinel-1 and Sentinel-2 satellite imagery. A total of 80% of images in the benchmark dataset were used as training set and 20% of the images were used as test set. the detected object belonging to the detected class. In a post-processing task, we retain only the detections that were classified with high-confidence (>0.6), and we geo-reference the coordinates of the detected objects by transforming them from topological coordinates, so that they can be correlated with the respective AIS coordinates. Hence, our results includes a list of detections that include the image tile that the detected object belongs to, its geo-referenced location (the geo-referenced centre of its bounding box), the acquisition time of the image. Different results were produced per vessel type detected (e.g., Tanker, Cargo and Tug for Sentinel-1 imagery). After we detect vessels in satellite images, we correlate this dataset with the respective AIS positions. The correlation task involves the following steps:  
1. _Spatio-temporal filtering_: The temporal resolution of satellite images is significantly lower than the temporal resolution of AIS messages (i.e., the revisit time of Sentinel satellites is 2-3 days in high coverage areas, whereas vessels with AIS transponders transmit AIS messages every few seconds or minutes, depending on their navigational status and speed). In order to be able to correlate these two data sources, for every image, we extract all AIS positions that are located into the area covered by the image during a 1 hour time window, spanning 30 min before and after the image acquisition time. For the filter, we create a temporal index on the geo-dataframe where we load all AIS positions, filtering out all positions that fall out of the time window; then, we perform spatial joins that retain only the positions that are covered by the spatial extent of the image. 2. _Interpolation_: Then, we create trajectories for each vessel contained in the dataset. For each trajectory, we retrieve the position of the vessel at the time the image was acquired by interpolating the vessel's locations before and after the acquisition time of the image. The output of this step is a snapshot of all AIS vessel positions which spatio-temporally intersect with the image. More specifically, we produce a file for each image, storing the position of every vessel located in the spatial extent of the image footprint at the time the image was acquired. 3. _Fusion:_ The fusion task matches the interpolated AIS-positions of the previous step with the vessels detected in the image that are the output of the vessel detection step and the post-processing step. Since the AIS dataset contains an identifier for each vessel, the fusion task assigns each detection to a vessel position. We perform a kNN-join between the two datasets in the following way: For each vessel detected in a satellite image, we search for the nearest AIS neighbor (i.e., nearest interpolated position of a vessel). To achieve this, we store the interpolated positions that are the output of the previous step in a KD-Tree [68] in order to speed up the distance joins.