# A Neural Network Approach to a Grayscale Image-Based Multi-File Type Malware Detection System
## 4 Results
### Comparison of Multi-File Type Models  
As per the results gathered from the previous experiments, both compact and regular neural network models are advisable for the selected application. Furthermore, it has also been found that the current dataset imbalance does not affect the model's performance. In this work, we investigate the potential benefits of merging datasets to enable the model to accurately identify malicious content in files regardless of the format, resulting in enhanced resilience and generalization capabilities of the combined model. The outcomes of these experiments are reported in Table 4, and the corresponding confusion matrices are presented in Figure 7. A total of 80% of the combined data is used for training, while 20% is used for testing purposes.  
As presented in Table 4 and Figure 7, the pre-trained models provided adequate and consistent results throughout the experiments. It should also be noted that a three-fold cross-validation technique was conducted in order to verify the robustness of these results. From these findings, it can be deduced that a compact neural network model can be utilized as a preferred solution. This offers high accuracy, which is comparable to what regular-sized models can provide, while also providing advantages in terms of resource requirements. For deployment purposes, having a smaller-sized model allows for a more compact overall size of the application.  
\begin{table}
\begin{tabular}{l l l l l l l} \hline \hline
**File Format** & **No. of Files** & **Train** & **Test** & **Model** & **Accuracy** & **Size** \\ \hline PDF & 19,889 & 15,912 & 3977 & AlexNet & 96.73\% & 227 MB \\ \hline MS Documents & 5770 & 4617 & 1153 & AlexNet & 87.44\% & 227 MB \\ \hline PE & 9952 & 7962 & 1990 & AlexNet & 92.60\% & 227 MB \\ \hline PDF & 19,889 & 15,912 & 3977 & GoogleNet & 96.12\% & 24 MB \\ \hline MS Documents & 5770 & 4617 & 1153 & GoogleNet & 86.35\% & 24 MB \\ \hline PE & 9952 & 7962 & 1990 & GoogleNet & 90.78\% & 24 MB \\ \hline \hline \end{tabular}
\end{table}
Table 3: Results summary for individual file type malware detection model.  
Figure 6: Confusion matrices for AlexNet model trainings using individual file types (L to R): Office, PDF, PE.  
It is worth mentioning that the multi-file type malware detection model has demonstrated an enhancement in overall accuracy. This improvement can be attributed to two key factors. First, the combination of all the data ensures a nearly equal distribution across the categories, as is evident in Table 4. This mitigates any bias toward larger datasets. Additionally, the preeminence of PDF files, which constitute the largest share of samples in the combined model, may contribute to the observed increase in overall accuracy.  
Since the main aim of these experiments is to deploy the model in a usable application, we further examined the reliability and performance of the trained model; aside from the testing sets, a detailed testing methodology was also applied. This tests the trained model against data that do not belong in the same category as the train or test sets, such as other related databases. This is detailed in the next sub-section.