# A Study on Detection of Malicious Behavior Based on Host Process Data Using Machine Learning
## 4 Experiments
### Parameters and Performance Evaluation of Models  
We conducted experiments by training the KNN, NB, RF, AE, and MemAE algorithms using the original dataset, the preprocessed dataset (ADASYN) and the preprocessed dataset (SMOTE). The parameters used for the training of the AE and MemAE algorithms are shown in Table 4.  
Figure 7: Data distribution plot of original dataset and preprocessed dataset (SMOTE).  
The purpose of this experiment is to detect malicious behaviors based on the behavior data that occur within the host. Such study can be defined as an anomaly detection problem. The following performance indicators are mainly used for such anomaly detection problems.  
1. Precision: It indicates the ratio of true positive samples among the samples classified as abnormal by the model. In other words, precision is a metric that aims to reduce False Positive (FP) cases. Higher precision means that the model correctly classifies most of the samples as abnormal.
2. Recall: It indicates the ratio of true positive samples among the actual abnormal samples. In other words, recall is a metric that aims to reduce False Negative (FN) cases. Higher recall means that the model correctly classifies most of the actual abnormal samples as abnormal.
3. F1-Score: It is the harmonic mean of precision and recall, representing the balance between precision and recall. Higher F1-score means that the model has both high precision and high recall.
4. Receiver Operating Characteristic (ROC) Curve: It is one of the ways to visualize the performance of the anomaly detection model. The ROC curve is a graph that shows the changes in True Positive Rate (TPR) and False Positive Rate (FPR) as the classification threshold of the model is changed. The better the model's performance, the closer the ROC curve is to the upper left corner, and the performance of the anomaly detection model is also expressed by the Area Under the Curve (AUC).  
These metrics are calculated based on the confusion matrix, as shown in Table 5, using Equations (14)-(18). In these equations, True Positive (TP) represents the number of cases that the model classified as abnormal and are truly abnormal, FP represents the number of cases that the model classified as abnormal but are normal, FN represents the number of cases that the model classified as normal but are actually abnormal, and True Negative (TN) represents the number of cases that the model classified as normal and are truly normal. TPR represents the ratio of true positives to actual positives, while FPR represents the ratio of false positives to actual negatives.  
\[\text{Precision}=\frac{\text{TP}}{\text{TP}+\text{FP}} \tag{14}\]  
\[\text{Recall}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{15}\]  
\[\text{F1}-\text{Score}=2\cdot\frac{\text{Precision}\cdot\text{Recall}}{ \text{Precision}+\text{Recall}} \tag{16}\]  
\[\text{TPR}=\frac{\text{TP}}{\text{TP}+\text{FN}} \tag{17}\]  
\[\text{FPR}=\frac{\text{FP}}{\text{FP}+\text{TN}} \tag{18}\]  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameters** & **AE Value** & **MemAE Value** \\ \hline Epochs & 100 & 1 \\ Batch size & 32 & 1 \\ Activation function & LeakyReLU, Linear & relu, sigmoid \\ Optimizer & adam & adam \\ Memory size & - & 100 \\ Loss function & Mean squared error & Mean squared error \\ \hline \hline \end{tabular}
\end{table}
Table 4: Parameters used in the experiment.