# Supervised Segmentation of NO2 Plumes from Individual Ships Using Troppomi Satellite Data
## 4 Results
### Plume Segmentation  
In Table 3, we report the results of the pixel classification based on a five-fold cross-validation for all models and benchmarks studied. Figures 11 and 12 provide the precision-recall and the ROC curves respectively. Both figures were obtained by averaging the scores over all cross-validation test sets. In Figure 13, we visualize the model coefficients for the linear models studied, as well as the impurity-based feature importance coefficients for the tree-based models (Random Forest and XGBoost). The obtained results can be summarized as follows:  
(i) From Table 3, Figure 11, as well as Figure 12, we can conclude that nonlinear classifiers clearly outperform both linear classifiers and threshold-based univariate benchmarks. Both used measures: AP score and ROC-AUC resulted in a similar rank of the studied classifiers. With XGBoost, Random Forest, or RBF SVM models, a very high level of precision can be achieved. For the task of ship plume segmentation, our biggest interest lies in the correct segmentation of the most representative pixels of the ship plume. Thus, the obtained level of recall we consider as reasonably satisfactory. From Table 3, we can also see that the level of the standard deviation of AP scores for multivariate nonlinear models is significantly lower than for linear or univariate models. This suggests that the results obtained with the nonlinear classifiers are more robust.  
(ii) From Figure 13, we can see that Linear SVM, Logistic Regression, Random Forest, and XGBoost multivariate models utilize the spatial information provided by sub-sectors and levels. The complexity of the RBF SVM model does not allow the direct calculation of the importance of the utilized features. Even though due to the different nature of the models, the coefficients' values depicted in Figure 13 cannot be compared directly, the relative differences between the models' features go along with our intuition on where the plume produced by an analyzed ship should be located within a _normalized sector_. For  
Figure 10: Nested cross-validation—illustration scheme.  
instance, high negative coefficients for the linear models that correspond to the features _Level 4_ and _Level 5_ suggest that even if a high-value pixel does occur in those regions of the _normalized sector_, it was most probably produced by a source other than the analyzed ship. On the other hand, the high positive coefficients corresponding to a feature _Sub-sector 2_ tell us that if a high-value pixel occurs in the middle of the sector, it is most probably a part of the plume produced by the studied ship.  
Figure 11: Precision–recall curve based on 5-fold cross-validation. Dashed lines indicate the results obtained from univariate threshold-based methods that, in this study, we considered as benchmarks.  
Figure 12: Receiver Operating Characteristic (ROC) curve based on five-fold cross-validation. Dashed lines indicate the results obtained from univariate threshold-based methods that, in this study, we considered as benchmarks.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Model** & **AP** & **ROC-AUC** \\ \hline Linear SVM & 0.609 \(\pm\) 0.063 & 0.935 \(\pm\) 0.009 \\ Logistic & 0.610 \(\pm\) 0.064 & 0.936 \(\pm\) 0.010 \\ RBF SVM & 0.742 \(\pm\) 0.031 & 0.951 \(\pm\) 0.008 \\ Random Forest & 0.743 \(\pm\) 0.030 & 0.952 \(\pm\) 0.008 \\ XGBoost & **0.745 \(\pm\) 0.030** & **0.953 \(\pm\) 0.007** \\ NO2 threshold & 0.375 \(\pm\) 0.062 & 0.823 \(\pm\) 0.017 \\ Moran’s \(I\) threshold & 0.493 \(\pm\) 0.063 & 0.912 \(\pm\) 0.011 \\ Moran’s \(I\) on high NO2 & 0.607 \(\pm\) 0.056 & 0.922 \(\pm\) 0.010 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Results on the test set with 5-fold cross-validation. Bold font indicates the best-obtained result. Under the dashed line: results obtained from univariate threshold-based methods that, in this study, we considered as benchmarks.