# Enhancing Malware Detection via RGB Assembly Visualization and Hybrid Deep Learning Models
## 5 Experimental Setup and Results
This remarkable improvement in accuracy is further supported by the best validation loss, which decreased dramatically from 0.0155 (Adam) to 0.0016 (AdaBoB) and an exceptionally low 0.0001 (AdaBelief). This suggests that these optimizers are more effective at navigating the loss landscape and guiding our model toward a more optimal minimum. While this performance gain comes with a modest increase in training time (approx. 14% longer than Adam), the substantial boost in predictive accuracy represents a highly favorable trade-off. In conclusion, this analysis confirms the generalizability of \begin{table}
\begin{tabular}{c c c c c} \hline  & **Precision** & **Recall** & **F1-Score** & **Support** \\ \hline Adposhel & 0.9434 & 1.000 & 0.9709 & 100 \\ \hline Agent & 0.9785 & 1.000 & 0.9891 & 91 \\ \hline Allaple & 0.9057 & 0.9500 & 0.9273 & 96 \\ \hline Amonetize & 0.9505 & 0.9100 & 0.9300 & 100 \\ \hline Androm & 1.000 & 1.000 & 1.000 & 100 \\ \hline Autorun & 1.000 & 1.000 & 1.000 & 99 \\ \hline BrowseFox & 1.000 & 1.000 & 1.000 & 99 \\ \hline Dinwod & 1.000 & 1.000 & 1.000 & 25 \\ \hline Elex & 0.9346 & 1.000 & 0.9662 & 100 \\ \hline Expiro & 1.000 & 1.000 & 1.000 & 100 \\ \hline Fasong & 1.000 & 1.000 & 1.000 & 100 \\ \hline HackKMS & 1.000 & 1.000 & 1.000 & 100 \\ \hline Hlux & 1.000 & 1.000 & 1.000 & 100 \\ \hline Injector & 0.9894 & 1.000 & 0.9946 & 93 \\ \hline InstallCore & 1.000 & 1.000 & 1.000 & 2 \\ \hline MultiPlug & 1.000 & 1.000 & 1.000 & 100 \\ \hline Neoreklami & 1.000 & 1.000 & 1.000 & 100 \\ \hline Neshta & 1.000 & 1.000 & 1.000 & 100 \\ \hline Regrun & 1.000 & 1.000 & 1.000 & 5 \\ \hline Sality & 1.000 & 0.9796 & 0.9897 & 98 \\ \hline Snarasite & 0.9615 & 1.000 & 0.9804 & 100 \\ \hline Stantinko & 0.9891 & 0.9100 & 0.9479 & 100 \\ \hline VBA & 1.000 & 1.000 & 1.000 & 100 \\ \hline VBKrypt & 0.9796 & 1.000 & 0.9897 & 48 \\ \hline Vilsel & 1.000 & 1.000 & 1.000 & 100 \\ \hline benign & 0.9947 & 0.9793 & 0.9869 & 965 \\ \hline accuracy & & & 0.9845 & 3121 \\ \hline macro avg & 0.9850 & 0.9827 & 0.9839 & 3121 \\ \hline weighted avg & 0.9848 & 0.9845 & 0.9845 & 3121 \\ \hline \end{tabular}
\end{table} 
Table 9: Comparison of malware families of MalevisAsm dataset by using RGB images. our method and demonstrates that its full potential can be unlocked with state-of-the-art optimization strategies. To validate the synergy of our proposed hybrid architecture, we conducted an ablation study by replacing the EfficientNet-B0 backbone with a standard ResNet-50 model. As shown in Table 11, this change resulted in significant performance degradation. The overall accuracy dropped from 98.50% to 57.64%, and more critically, the macro average F1-score decreased from 98.39% to a mere 49.54%. The alternative model completely failed to detect certain families (e.g., Autorun, VBKrypt) and exhibited poor precision or recall for many others. This empirical evidence strongly supports our hypothesis that the compound scaling and efficient fine-grained feature extraction of EfficientNet-B0 are crucial for capturing the subtle, discriminative patterns in our RGB-encoded malware representations, a task for which ResNet-50 is less suited. This confirms that the chosen pairing is not arbitrary but a key component of our system's success.