## 6 Results
### Comparison of Metrics  
To facilitate comparison between the models, the metrics for each model are charted in Figure 5, and false positive and false negative rates are charted in Figure 6.  
The KNN model does not achieve high values for accuracy, true positive rate, true negative rate, precision, or \(F1\) score. While the values of these metrics for the SVM model are higher, both the SVM and KNN models have a high false-negative rate, making them unsuitable for the classification of ransomware. Figure 5: Accuracy, TPR, TNR, precision, and F1-score. Figure 6: False-positive and false-negative rates. The Logistic Regression model scores slightly lower than the SVM model for accuracy, TPR, TNR, precision, and \(F1\) score. However, the false-negative rate for logistic regression is higher than that of the SVM model. Because of these factors, it is a worse classifier choice than SVM. The Decision Tree model achieves the highest accuracy and \(F1\) score, with a false-negative rate of only 0.65%. This would make it a good choice as a classifier, but the tendency of decision trees to overfit to training data must be weighed against this. Random Forest achieves the best true positive rate, but the values for accuracy, true negative rate, precision, and \(F1\) score are unexceptional. However, it should be noted that the Random Forest model produced a false negative rate of 0.00%. ANN Model 3 achieved good values for accuracy, true negative rate, precision, and \(F1\) score. It is in the top two scores across most of the metrics shown in Figure 5. While this indicates successful classification, the false-negative rate of 4.58% is a concern. The Net Benefit for a range of probability thresholds, calculated according to [PERSON] et al. 's formula [44], is plotted for each model in Figure 7. All have a positive Net Benefit for values of \(p\leq 0.07\), with ANN Model 3 and the logistic regression model providing the greatest Net Benefit. If the models were to be judged on accuracy alone, it would appear that the Decision Tree and ANN Model 3 are the most capable, with accuracies of 97.61% and 96.93%, respectively. The \(F1\)-score is a more useful metric since it is based on recall and precision values. The \(F1\)-scores for these two models are also good, at 97.75% and 97.01%, respectively. If Net Benefit were to be used as the sole criterion, it would appear that ANN Model 3 and the logistic regression model would give the best classification performance. However, the consideration of other metrics reveals more about the characteristics of each model, which could influence their suitability for detecting ransomware. The worst prediction a ransomware detection system could make would be a false negative, i.e., to incorrectly classify ransomware as non-malicious. The comparatively high false-negative rates for the logistic regression (6.54%) and the ANN Model 3 neural network (4.58%) models suggest that these models should be used cautiously to detect ransomware (Figure 6). Figure 7: Net benefit of each model. Due to the critical nature of the task of identifying ransomware, and the negative consequences of a false negative classification, a more 'cautious' model may be preferred. While the random forest model does not have the highest accuracy or precision, the model's True Positive Rate of 100% and False Negative Rate of 0% suggest that its classifications are more 'cautious' than classifications produced by the other models. It has a tendency to classify benign samples as ransomware and appears unlikely to classify ransomware as benign. For this reason, the random forest model might be considered the'safest' of the models evaluated for the task of ransomware identification. This illustrates the risks of relying solely on metrics to judge the suitability of a model without considering the nature of the classification problem, the risk appetite of those interpreting the prediction, and the consequences of false negatives in particular.