## 3 Proposed Method
### Dataset Preparation  
The 'CryptoRansom' archive [35] was the source of 38,157 samples, each named after its MD5 hash. Overall, 730 Malware samples were submitted to Cuckoo in batches using the web interface. To avoid class imbalance problems [26], 735 benign samples were also submitted. Benign samples were gathered from a Windows environment in the form of Executables and DLLs. The results from each sample were logged into a directory structure and a MongoDB database. The JSON files produced by Cuckoo were of particular interest due to the nature and quantity of information they contained. Each JSON file's 'apistats' section had information about processes spawned by the sample, which process made API calls, and how many times each API function was called.  
A bash script was used to rename all the report.json files to <task_id>.json, and copy them to a single directory. The files in this directory were then compiled into a compressed archive using tar and gzip. The resulting archive file was then transferred to the computer used for machine learning work, unzipped, and extracted.  
A MariaDB database was created to contain the statistics on API call frequency. The database schema is shown in Figure 3. This schema model shows that each sample spawned one or more processes, and each process made multiple API calls. Using a relational database allows the creation of different datasets using different queries. For example, it would be possible to produce a dataset containing information only on DLL samples or only for samples that spawn multiple processes. The database was populated from the collected JSON log files by a python script. Once populated, the record counts for each table were as shown in Table 2.  
A query to aggregate the API calls made by the processes spawned by each sample was run, and the results were output to a CSV file. A code example for the query is:  
SELECT s.sample_id, s.is.malicious,  
COUNT(p.process_id AS process_count, a.api_id,  
SUM(a.call_count) AS call_count  
FROM malware_stats.sample s  
JOIN malware_stats.process p ON s.sample_id = p.sample_id  
JOIN malware_stats.process_api_call a ON p.process_id = a.process_id  
GROUP BY s.sample_id, s.is malicious, a.api_id  
The number of times malicious and benign samples called each API function was more than 565,902. Malicious samples called a wider range of API functions than benign samples, and overall malicious samples made more API calls than benign samples (8,607,508-929,971). The API functions most frequently called by malicious samples  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Table** & **Record Count** \\ \hline Sample & 1465 \\ Process & 6341 \\ process\_api\_call & 269,918 \\ api\_function & 284 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Database table record counts.  
Figure 3: Entity-relationship diagram of the database used to store API call profits.  
were: LdrGetProcedureAddress, ReadProcessMemory, NtClose, RegQueryValueExW, and NtReadFile; for benign samples, they were: NtReadFile, SetFilePointer, GetSystemMetrics, NtClose, and LdrGetProcedureAddress.  
The API usage statistics were used to classify the examples as either benign or malicious. This data was re-arranged into a format suitable for training machine learning models by a script (csv_to_dataset.py) that uses pandas' pivot table functionality. The resulting dataset contained 1465 labelled samples with the label 'is_malicious'.  
Each sample had 284 features, and each feature was the number of times the associated API function was called. The mentioned 284 features have integer values; therefore, no transformation of categorical features using one-hot encoding was necessary.