## 5 Results
### Model Performance  
For Model 1, a single-class binary output type of model was considered. To apply the supervised learning algorithms, 75% of the data set was used for training, and 25% for testing and validation, this division being heuristically enough to find good solutions for linear problems. After both algorithms are trained, a confusion matrix with the validation set is obtained for the respective algorithm implementation, and the corresponding performance criteria are determined (see Figure 5). For each algorithm, the model can be validated by the MLP's binary cross entropy function, which is computed between true and predicted labels, and the Precision-Recall curve for RF. For the MLP's loss function, its behavior throughout the epochs can help determine the fit of data set, which helps shaping its structure. Both training and validation curves have an exponential decay over the first epochs, which means that the MLP quickly finds a good fit and does not seem to overfit, as the validation cross entropy curve does not seem to increase and the training curve decreases. Given the characteristics of the data set for this model, the behavior of the Precision-Recall curve evaluates the skill of the prediction, as the graph computes different thresholds for precision and recall. The area under the curve for Model 1 with RF is significant, which helps to judge favorably the quality of the prediction, thus validating the classifier. Table 4 shows that the RF is better at correctly predicting the positive class than the MLP, because, while it is possible to observe that both algorithms are similarly capable of detecting the predominant class (No Fatality), not only the RF algorithm is able to correctly identify the Fatality class more often than the MLP, but it also does so without predicting fatal occurrences as non-fatal. Semi-supervised learning was used with a pool-based strategy, but no improvement over the other algorithms resulted, because the amount of data is too small for this algorithm to reach a new perspective. Furthermore, with a lower score on an important metric such as recall, AL does not seem to suit this model well. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Predicted** & **True Class** & \\
**Class** & **0** & **1** \\ \hline
0 & True Negative & False Negative \\ \hline
1 & False Positive & True Positive \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Confusion matrix for binary classification. For Model 2, there is even less data, since only the accidents with fatalities are considered. Thus, MLPs face limitations due to the error function for the first epochs being very high (Figure 6b). Despite the good shape of the loss function, the loss decay of the training data set is severe, which makes it difficult to judge what is regarded as overfit or underfit of the model, and the behavior of both curves as the epochs progress. The accuracy function better demonstrates what the evolution of MLP with more training is (Figure 6a), as the training curve is consistent with a regular accurate curve, and so the data structure was considered to be well fitted. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Type of Learning** & & & **Precision** & **Recall** & **F1-Score** & **Accuracy** \\ \hline Random & Class & No Fatality & 0.92 & 0.94 & 0.93 & \\ Forest & Averages &
\begin{tabular}{c} \\ \end{tabular} & 0.84 & 0.77 & 0.80 & \\  & & Macro & 0.88 & 0.86 & 0.87 & \\  & & Weighted & 0.89 & 0.90 & 0.89 & \\ \hline Multilayer & Class & No Fatality & 0.89 & 0.92 & 0.90 & \\  & & Fatality & 0.75 & 0.59 & 0.66 & \\  & & Macro & 0.80 & 0.76 & 0.77 & \\  & & Weighted & 0.83 & 0.83 & 0.83 & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Classification Report for Model 1. Figure 5: Confusion Matrixes for Model 1 (left) and Model 2 (right) with RF algorithm (top) and MLP algorithm (bottom). (**a**) Random Forest, Model 1. (**b**) Random Forest, Model 2. (**c**) Neural Network, Model 1. (**d**) Neural Network, Model 2.