## 4 Data and Evaluation Metrics  
This evaluation study was conducted by considering a collection of 27,035 Windows PE files: 13,494 goodware and 13,541 malware. Notably, we made the dataset we prepared to conduct this evaluation study publicly available for future research (see the Data Availability Statement of this article). Goodware was selected from the public PEMML repository, which is the only online repository that we have found to publicly provide binaries of goodware for free. Specifically, we selected 980 goodware files recorded in 2017 and 12514 goodware files recorded in 2018. Malware files were selected from VirusShare. In particular, we randomly selected 6459 malware files recorded in 2021, 83 in 2022 and 6999 in 2023. The following three queries were performed on VirusShare to retrieve the Windows PE malware files:  
1. \"filetype: \"PE32 executable\" extension:exe after 1 January 2021\",
2. \"filetype: \"PE32 executable\" extension:exe after 1 January 2022\",
3. \"filetype: \"PE32 executable\" extension:exe after 1 January 2023\". The queries were performed between March and June 2023. Malware files were downloaded in that period one by one. Notably, the dataset used is larger and contains more recent Windows PE files than the dataset used in [5], which considered 104 Windows PE malware files retrieved from DasMalwerk in late 2018. A short description of the PEMML and DasMalwerk repositories, as well as the VirusShare service, is reported in Section 2.2. We selected the first 1 Mb of each binary Windows PE file to be processed with MalConv. According to the description reported in [6], for each Windows PE file whose size was smaller than 1 Mb, the embedded input representation of this file was padded with the embedded value \(\Phi(256)\). We recall that, as described in Section 3.2.1, MalConv applies the byte-embedding transformation \(\Phi(\cdot)\) to transform each integer value recorded in a byte into 8-dimensional real embedding that feeds the embedding input space of a fixed size of MalConv. Therefore, whenever the Windows PE file size is smaller than 1 Mb, the sequence of the 8-dimensional real embeddings that were generated from the sequence of bytes actually recorded in the binary file is padded with the 8-dimensional real embedding of the value 256 until the expected size of the embedding input space of MalConv is reached. In addition, we used the static analyser of LIEF (version 0.9) ([[https://github.com/lief-project/LIEF](https://github.com/lief-project/LIEF)]([https://github.com/lief-project/LIEF](https://github.com/lief-project/LIEF)), accessed on 30 April 2024 ) to perform the static analysis of each binary Windows PE file and extract the vector of 2381 engineered features to be processed with LGBM. We measured the performance of both MalConv and LGBM models by computing standard metrics commonly used to evaluate the predictive ability of binary classification models. Specifically, let us consider tm--the amount of Windows PE malware that is correctly predicted as malware; fm--the amount of Windows PE goodware that is wrongly predicted as malware; tg--the amount of Windows PE goodware that is correctly predicted as goodware; and fg--the amount of Windows PE malware that is wrongly predicted as goodware. We computed the following metrics:  
* _Overall accuracy_ (oa) measures the proportion of correctly classified Windows PE files, regardless of the class, out of all the predicted files, i.e., \(\text{oa}=\frac{\text{tm}+\text{tg}}{\text{tm}+\text{tg}+\text{tfm}+\text{tg}}\). This metric estimates the overall ability of a decision model to correctly classify a sample in its proper class, regardless of the class value. * _Precision_ (prec) measures how many Windows PE malware files are correctly classified as malware, given all predictions of the malware class, i.e., \(\mathsf{prec}=\frac{\mathsf{m}}{\mathsf{tm}+\mathsf{fm}}\). This metric estimates how often the decision model is correct when predicting the target class \"malware\".