# A Machine Learning Predictive Model for Ship Fuel Consumption
## 3 Analysis of Results

Table 7: Performance metrics of the winning model CatBoost. It is inferred that the model performed very well during the training phase, given the concentration of records around the line positioned at the origin and the optimal Coefficient of Determination (R\({}^{2}\)) equal to 0.991. However, it is possible to notice some outliers, predictions far from the actual value observed, that generate high residuals. The same happens when considering only the predictions during the test phase in which there is a higher number of residuals, impacting a lower Coefficient of Determination (R\({}^{2}\)), 0.912, indicating a loss of model performance; however, within normality, since they are data never before seen by the model and caused by means of the Cross Validation Method, avoiding model bias [32]. It can also be observed from Figure 6 that the model had difficulty in predicting records from the range of 4000 L, having generated, even in the test phase, three of the highest residuals. As a consequence, this data can certainly influence the increase in metrics such as RMSE, which penalizes large errors between prediction and observed reality [84]. The distribution of the records presented in Figure 7 was made dependent on each actual value of the observed record (\(x\)-axis) and the estimates provided by the model during the training phase (\(y\)-axis). The black dashed line represents a linear regression best suited to estimate the correlation between the predicted and measured value of the target variable. The dashed line in gray, called identity, outlines a function where y = x, forming 45 degrees, just to highlight whether the model is over or underestimated for the values analyzed [35]. The accumulation of predictions along the dashed lines for the range near the origin determines that the model has an assertive capacity for vessels that consume more than 2000 L. It is clear from Figure 7 that after this range, a larger number of outliers occur, that is, predictions with high residuals. This is caused because:  
The share of vessels consuming more than 2000 L is small compared to the rest of the database (about 29.3% of the amount), leaving the model more susceptible to errors in the case of vessels with discrepant dimensions than those contained in large numbers;  
These outliers may be an indication of fuel consumption beyond what is normal for vessels of this size, evidencing fraudulent operational behavior. For the final validation of the model developed through the hold-out validation method, records were randomly chosen and separated from the initial database to assess the performance of the model against the new input data. A total of 125 vessels were  
Figure 6: Existing residual curve between predicted and actual observed values. separated, 5 of which are listed in Table 8. The numbers in the first row indicate the model's input predictors as well as the target variable. Faced with the new data, Table 9 shows the main performance parameters. The Coefficient of Determination reached was 0.910, thus demonstrating its high performance even for records never seen before.