# A Novel Malware Detection Model in the Software Supply Chain Based on LSTM and SVMs
## 5 Experimental Results and Analysis
### Experimental Results and Analysis
#### 5.4.1 Evaluation of BO-SVM Efficiency  
This paper builds four additional models to validate the performance of the BO-SVM model, which are as follows:  
1. Basic SVM model: SVM model with default parameter settings.
2. ACO-SVM model: SVM model optimized based on the Ant Colony Optimization (ACO) algorithm.
3. PSO-SVM model: SVM model optimized based on the PSO algorithm.
4. BO-DT model: A Decision Tree (DT) model optimized based on the BO algorithm.  
The SVM model uses the default values. The initial parameters of the other models were adjusted to the values in studies [31; 32; 33], respectively. Afterwards, the parameter combinations that made the model perform best were determined after several experiments. See Table 3 for the detailed parameter settings.  
With the purpose of investigating its generalization ability and adaptability, the proposed BO-SVM model was compared with the above four models, i.e., ACO-SVM, PSO-SVM, BO-DT, and SVM, on the ClaMP dataset and the CICMalDroid-2020 dataset. To further verify the performance of the BO-SVM model, the experimental results on the ClaMP dataset were especially compared and analyzed with the related work of [PERSON] et al. [34]. Table 4 displays the outcomes of the experiment.  
From Table 4, it can be observed that the accuracy of the BO-SVM model outperforms the other models on both datasets. The SVM model, optimized by the BO algorithm, outperforms the SVM models optimized by the ACO and PSO algorithms for the two datasets mentioned above. This is because the ACO-SVM and PSO-SVM models have the problem that the algorithms may fall into local optimality when optimizing the hyperparameters \(\gamma\) and \(C\), whereas the basic SVM model relies on empirical manual parameter settings, which may not be able to guarantee that the global optimal solution is reached. In addition, the BO-SVM model outperforms the BO-DT model and the KNN model reported in reference [34].  
In terms of training time, the basic SVM model has the shortest training time, while the BO-SVM model and BO-DT model have similar training times, but both are longer than the basic SVM model. This is mainly due to the fact that the introduction of the BO algorithm increases the training time accordingly, but the increase is still within the acceptable range.  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{3}{c}{**Accuracy (\%)**} & \multicolumn{3}{c}{**Training Time (in Seconds)**} \\ \cline{2-5}  & **ClaMP** & **CICMalDroid-2020** & **ClaMP** & **CICMalDroid-2020** \\ \hline BO-SVM & 95.3 & 96.4 & 34.6 & 40.5 \\ ACO-SVM & 95.0 & 96.0 & 5185.4 & 5434.5 \\ PSO-SVM & 94.8 & 95.8 & 4223.4 & 4734.2 \\ BO-DT & 95.1 & 95.7 & 32.4 & 35.4 \\ SVM & 87.5 & 94.3 & 1.7 & 3.1 \\ [PERSON] et al. [34] KNN & 93.5 & \(\backslash\) & \(\backslash\) & \(\backslash\) \\ \hline \hline \end{tabular}
\end{table}
Table 4: The accuracy and training time of each model on the two datasets.  
\begin{table}
\begin{tabular}{c c c} \hline \hline \multicolumn{1}{c}{**Model**} & **Parameter** & **Value** \\ \hline BO-SVM & Kernel & RBF \\ \hline \multirow{8}{*}{ACO-SVM} & Kernel & RBF \\  & Art Count & 30 \\  & MaxIter & 100 \\  & Pheromone factor & 3 \\  & Pheromone constant & 500 \\  & Heuristic function factor & 4 \\  & Pheromone volatilization & 0.3 \\  & factor & \\ \hline \multirow{8}{*}{PSO-SVM} & Kernel & RBF \\  & Particle Size & 30 \\  & MaxIter & 100 \\  & Inertia Weight & 0.9 \\ \cline{1-1}  & Acceleration Coefficients 1 & 2 \\ \cline{1-1}  & Acceleration Coefficients 2 & 2 \\ \cline{1-1}  & Velocity Limits\_min & \(-5\) \\ \cline{1-1}  & Velocity Limits\_max & 5 \\ \hline \multirow{2}{*}{SVM} & Kernel & RBF \\ \cline{1-1}  & \(\gamma\) & 5 \\ \cline{1-1}  & C & 1.0 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Parameter settings of each model.  
However, the ACO-SVM and PSO-SVM models not only take longer to train, but also are less accurate than the BO-SVM model.