# Enhancing Malware Detection via RGB Assembly Visualization and Hybrid Deep Learning Models
## 3 Materials and Methods
### Dataset Description  
The dataset contains malware samples of different numbers and families. The state of these samples before conversion is in Portable Executable (PE) form. This study explores the potential of classifying malware by extracting semantic patterns from code and representing them as images. After validating the methodology with an initial experiment, a deep learning-based classifier was employed to enhance performance. A new dataset, MalevisAsm, was created by combining the MaleVis dataset with benign files to improve the model's generalization in detecting various malware threats. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Approach** & **Strengths** & **Limitations** & **Our Solution** \\ \hline Binary-to-grayscale & Simple visualization & Loss of semantic info & Assembly-to-RGB \\  & & & with opcode context \\ \hline Single CNN models & Fast processing & Limited feature diversity & Hybrid CNN \\ \hline Traditional ML & Interpretable & Poor generalization & Deep learning with UMAP \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparative analysis and research gaps. The MaleVis (Malware Evaluation with Vision) dataset, publicly released in 2019 by [PERSON] et al. [32], contains 14,226 RGB images. The MaleVis dataset, which is available through the link ([[https://web.cs.hacettepe.edu.tr/~selman/malevis/](https://web.cs.hacettepe.edu.tr/~selman/malevis/)]([https://web.cs.hacettepe.edu.tr/~selman/malevis/](https://web.cs.hacettepe.edu.tr/~selman/malevis/)) accessed on 22 June 2025) provided by the authors, contains only RGB images. In this study, malware exe files based on the MaleVis dataset were used and a different dataset was obtained from the original MaleVis dataset. Although the MaleVis dataset consists of images, it is derived from malware exe files and is therefore domain-specific. The dataset provides 25 malware classes. The benign class was created by the aggregation method. The benign executable files in our dataset were sourced from two main avenues to ensure both diversity and reliability. One portion was collected from our own clean and actively used Windows systems, including installed software utilities and default system files. The other portion was obtained from trusted and legally permissible open-source platforms such as SourceForge, GitHub between [Start Date January 2025] and [End Date March 2025], and official vendor websites, with an emphasis on widely used, signed, and open-source applications. To verify their benign nature, all files were scanned using multiple reputable antivirus engines, including Windows Defender, and only those with zero detections across all scans were retained. Each file was also tested in sandboxed environments like Windows Sandbox, and those showing any alerts or suspicious behavior were excluded. Additional behavioral checks using tools such as Windows Defender Application Guard were performed for further validation. Files with conflicting antivirus results, rare detection flags, or unsigned origins were removed to reduce the risk of false positives and ensure dataset integrity. In this way, a total of 26 classes were obtained. It contains 15,212 images: 12,091 training images and 3121 validation images. All images are resized to 224 \(\times\) 224 pixels for uniformity. Table 4 provides example RGB images from the Malevis dataset, highlighting the visual differences between malware classes. Also, Table 5 presents the detailed class distribution of the MalevisAsm dataset. As shown, the dataset exhibits class imbalance with some classes containing significantly fewer samples (e.g., InstallCore.C: 8 samples) compared to others (e.g., Adposhel: 500 samples). This imbalance reflects the real-world distribution of malware families, where certain variants are naturally more prevalent.