## 2 Methodology
### Performance Metrics  
In this paragraph, we describe the performance metrics that were used to evaluate the performance of this study. In general, the performance metrics are used in order to provide a common measure of the performance of the trained classifier, against new images from the testing set. The outcome of this prediction, in comparison to the actual class label that was assigned to the image, can take one of the four values, true positive (TP) or true negative (TN) if it is classified correctly, and false positive (FP) or false negative (FN) if it is misclassified.  
These values are then used to calculate the performance metrics that are most commonly used in classification problems. In Table 2 we present the performance metrics used in our study for evaluation of the performance of our classifier, together with their descriptions, as well as their mathematical formula.  
Additional to the performance metrics, we choose to visualize the prediction results in confusion matrices. The confusion matrix is a table that displays the aforementioned values in such a way that one can easily view the number of properly classified examples, as well as false positives and false  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Dataset** & **Training** & **Validation** & **Testing** \\ \hline \% of total dataset & 68\% & 17\% & 15\% \\ \# of images & 3053 & 764 & 674 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Data splitting percentages and purpose.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Name** & **Description** & **Formula** \\ \hline \multirow{2}{*}{_Accuracy_} & ratio of correctly predicted observation to the & \multirow{2}{*}{(TP + TN/TP + FP + FN + TN)} \\  & total observations (preferred in balanced & \\  & datasets) & \\ \hline \multirow{2}{*}{_Precision_} & ratio of correctly predicted positive observations & \multirow{2}{*}{(TP/TP + FP)} \\  & to the total predicted positive observations & \\ \hline \multirow{2}{*}{_Recall_} & ratio of correctly predicted positive observations & \multirow{2}{*}{(TP/TP + FN)} \\  & to all observations in actual class & \\ \hline \multirow{2}{*}{_F1 score_} & is the weighted average of Precision and Recall & \multirow{2}{*}{[2 \(\cdot\) _Recall \(\cdot\) Precision]_[_Recall \(\cdot\) _Precision_] \\  & (preferred in unbalanced datasets)} & \\ \hline \hline \end{tabular}
\end{table}
Table 2: Performance metrics used in our study.  
negatives. In our study, which is a binary classification, the confusion matrix is of size \(2\times 2\). The template used for the confusion matrix is shown in Table 3.  
All the confusion matrices with the results for each tested scenario are located in the Appendix A of this paper.  
Finally, a loss function (or objective function, or cost function) is used to evaluate how well the specific algorithm performs on the given data. In our study, we used the mean squared error (MSE), where the average squared difference between the estimated values and the real values is calculated during the training. It always has non-negative values and it is preferred, because large errors create larger consequences than equivalent smaller errors.  
The value of the loss function is important for the evaluation of the model's performance, because it can show us if the model can improve its performance and how well the predictions correspond to the real values. The closest that this value is to 0, the better will be the performance of the model.