# Is Malware Detection Needed for Android TV?
## 4 Performance Results
### The Performance of the Model Trained with TV Applications on TV Applications  
We used 103 TV applications from the Androzoo Payload-Injected TV dataset for the feature selection. Then, we trained the classification models on the remaining 935 TV applications in the Androzoo Payload-Injected TV dataset. We tested the performance of the trained models on 370 TV applications in the APK Mirror Payload-Injected TV dataset. We trained and tested the models on MATLAB Online 2024b, which was run on a machine with an Intel Xeon Platinum 8375C CPU @ 2.90 GHz (Santa Clara, CA, USA) without parallel computing. We measured the training and testing times for each model, which are presented in Figure 4 and Figure 5, respectively. According to our observations, four models (tree, svm, knn, and discr) performed training in less than 1 millisecond on average per application. The svm and tree models showed a close performance, at 0.3511 and 0.3443 milliseconds, respectively. The worst training time was achieved by the kernel classification model, with an average of 30 milliseconds per application. The performances of the four models (tree, net, linear, and discr) were distinguishing with respect to the other models in testing. We observed that the tree and net models performed similarly, with 0.1149 and 0.1152 millisecond testing times per application on average. The performance of the knn model was poor in testing, which completed testing in 0.9002 milliseconds on average per application. We calculated the confusion matrix for each model on the test set. The confusion charts for the models are presented in Figure 6, where benign and malicious applications  
Figure 4: Average training time of models per application. are labeled with the 0 and 1 class labels, respectively. The knn model predicted 31 malicious and 15 benign applications incorrectly, which was not good. On the other hand, the discr model misclassified only two malicious and six benign applications. Similarly, the ensemble model missed five malicious and four benign applications. We present the Receiver Operating Characteristic (ROC) curve for each model in Figure 7, which visualizes the performance across all of the thresholds using the True Positive Rate (TPR) and False Positive Rate (FPR) metrics. In our case, the area under the ROC curve (AUC) represents the probability that the model will correctly classify a randomly chosen malicious application over a randomly chosen benign application. With respect to the AUC metric, the discr and ebesemble models performed well, with 0.9956 and 0.9945 scores, respectively. The worst performance was obtained by the tree classifier, with an AUC score of 0.8847. Figure 5: Average testing time of the models per application. Figure 6: Confusion charts for classifiers. According to the F1-Score and MCC metrics, the best result was achieved by the discr model, with 0.9789 and 0.9570 scores, respectively. A similar performance was obtained by the ensemble, net, and svm models, with 0.9760, 0.9735, and 0.9705 scores with respect to the F1-Score metrics, respectively. The performance of the knn model was calculated as a 0.8722 F1-Score and a 0.7545 MCC score, which was the worst performance on the test set with respect to that of the other models. The F1-Score and MCC results for the models on the testing set are presented in Figures 8 and 9. The discriminant analysis classifier achieved the best results on the test set with an F1-Score of 0.9789 and an MCC score of 0.9570. It performed the fourth best in terms of the training time, at 0.8990 milliseconds per application. In testing, it classified malware in 0.1346 milliseconds on average, which was only 0.0197 milliseconds slower than the fastest model. The ensemble classifier  
The binary decision tree classifier was the third worst model on the test set, with an F1-Score of 0.9140 and an MCC score of 0.8872. On the other hand, it was the fastest model in both training and testing. On average, it spent 0.3443 milliseconds per application on the training set.