# The Use of Machine Learning Methods in Road Safety Research in Poland
## 2 Materials and Methods
### Evaluation of the Quality of the Classification Models  
Various metrics are used to evaluate classifiers. Typically, these metrics are derived from the confusion matrix, which evaluates the effectiveness of classification models in making predictions based on test data and indicates the performance of the classification model. This matrix can be used to compute various parameters of the model, such as the accuracy, precision, and sensitivity, and to plot the AUC-ROC curve [68]. A simplified confusion matrix with calculations of the basic metrics is shown in Figure 1[69]. The rows of this matrix represent the correct decision classes, while the columns represent the decisions predicted by the model. The value \(N_{ij}\) at the intersection of row \(i\) and column \(j\) determines the number of observations of class \(i\) classified into class \(j\), \(1\leq i,j\leq h\). In a general context, it takes the form presented in Table 1.  
Figure 1: Confusion matrix with calculations of the basic metrics. For each class, we estimated the basic values: _TP_ (true positive)-number of outcomes (instances) correctly classified as belonging to a particular class, _FP_ (false positive)-number of outcomes incorrectly classified as belonging to a particular class, _FN_ (false negative)-number of outcomes incorrectly classified as not belonging to a particular class, _TN_ (true negative)-number of outcomes correctly classified as not belonging to a particular class. According to the notation presented in Table 1 for the _j_-th class (\(1\leq j\leq h\)), the basic values were determined as follows:  
\[TP=N_{jj}, \tag{1}\]  
\[FP=\sum_{i=1}^{h}N_{ji}, \tag{2}\]  
\[i\
eq j\]  
\[FN=\sum_{i=1}^{h}N_{ji}, \tag{3}\]  
\[i\
eq j\]  
\[TN=\sum_{i,j=1}^{h}N_{ij}-TP-FP-FN. \tag{4}\]  
Additionally, for each class, we estimate the following basic metrics [70]:  
* Sensitivity (recall, true positive rate-_TPR_), indicating the extent to which a true positive class was classified as positive: \[TPR=\frac{TP}{TP+FN}\] (5)
* Specificity (true negative rate-_TNR_), indicating the extent to which a true negative class was classified as negative: \[TNR=\frac{TN}{TN+FP}\] (6)
* Positive predictive value (_PPV_), indicating the confidence in positive predictions, i.e., the percentage of positive predictions confirmed as true positives: Figure 1:0 (7)
* Negative predictive value (_NPV_), indicating the confidence in negative predictions, i.e., the percentage of negative predictions confirmed as true negatives: Figure 1:1 (8)
* Balanced accuracy is an arithmetic average of sensitivity and specificity, and it determines the average number of predictions for each class correctly classified by the model (particularly useful when there is only one test set and it is not balanced): \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Actual Class \(\rightarrow\)1** & **Class 1** & **Class 2** & **—** & **Class _h_** \\
**Predicted Class \(\downarrow\)2** & & & & \\ \hline Class 1 & \(N_{11}\) & \(N_{12}\) &... & \(N_{1h}\) \\ Class 2 & \(N_{21}\) & \(N_{22}\) &... & \(N_{2h}\) \\... &... &... &... &... \\ Class \(h\) & \(N_{h1}\) & \(N_{h2}\) &... & \(N_{hh}\) \\ \hline \hline \end{tabular}  
* Horizontal arrow refers to rows (Actual Class); 2 Vertical arrow refers to columns (Predicted Class).  
\end{table} 
Table 1: Confusion matrix form. \[Balanced\ Accuracy=\frac{TPR+TNR}{2}. \tag{9}\]  
For the entire classifier, we determined the accuracy (\(ACC\)), which denotes the fraction of all instances that are correctly categorized [71]:  
\[ACC=\frac{\sum_{i=1}^{h}N_{ii}}{\sum_{i,j=1}^{h}N_{ij}} \tag{10}\]  
Additionally, [PERSON]'s Kappa statistic was used to evaluate the classifiers. This metric is more informative than the accuracy when working with imbalanced data [72]. The Kappa is a measure of how closely instances classified by a machine learning classifier match the data labeled as ground truth, controlling for the accuracy of a random classifier as measured by the expected accuracy.