# Malicious JavaScript Detection Based on Bidirectional LSTM Model
## 5 Experiment and Result
### Detection Performance  
Answer to RQ1: As shown in Table 9, the BLSTM neural network achieves a good performance. The accuracy of the BLSTM neural network reached 97.71%, which indicates that most malicious samples can be accurately found during the test. At the same time, the model also achieved a recall rate of 97.91%, which means that it caused very few false positives. The F1-score is the harmonic mean of the precision and recall, where an F1 score reaches its best value at 1. It is one of the most important measurements for evaluating a model's performance. The F1-score of the BLSTM neural network reached a very high value of 98.29%, which means our model achieves higher rates of true positives with lower rates of false positives. Therefore, we conclude that the BLSTM neural network is extremely effective in detecting malicious JavaScript. Answer to RQ2: In order to compare with other malicious code detection approaches, we chose three widely used machine learning models, a malicious JavaScript detection tool and an antivirus software. As shown in Table 9, although the Naive Bayes algorithm achieves a relatively high recall rate, its accuracy is very low, only 64.61%. This means that a large number of benign samples are misclassified. If the Naive Bayes model is used to detect malicious JavaScript on the Internet, then much manpower must be invested to manually audit the detection results of the model to reduce the impact on normal websites. Contrary to the Naive Bayes model, the accuracy rate of the Support Vector Machine (SVM) model is higher than the recall rate, and the F1-score of 81.68% seems to perform well. However, we noticed that the recall rate of the SVM model is only 72.39%, which means that 27.61% of the malicious JavaScript evade the detection of the model. Therefore, such a model is not capable of detecting malicious JavaScript. We consider that the reason for the poor performance of these two models is that the models are too simple to fully describe the malicious features. The performance of random forest is much better than the previous two models, but it is still worse than the BLSTM neural network with 7.7% lower accuracy, 5.88% lower recall rate, and 7.17% lower F1-Score. We need to emphasize that malicious JavaScript detection is different from other neural network applications, such as speech recognition and emotion classification. Undetected malicious JavaScript may cause \begin{table}
\begin{tabular}{c c} \hline \hline
**Stage** & **Time Consumption (ms)** \\ \hline Learner & 0.241 \\ \hline Classifier & 2.763 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: The time consumption to learn the BLSTM neural network. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Methods** & **Accuracy** & **Precision** & **Recall** & **F1-Score** \\ \hline BLSTM & **97.71\%** & **98.68\%** & **97.91\%** & **98.29\%** \\ \hline Naive Bayes & 64.61\% & 67.91\% & 95.31\% & 79.31\% \\ \hline SVM & 81.62\% & 93.71\% & 72.39\% & 81.68\% \\ \hline Random forest & 90.01\% & 92.83\% & 89.53\% & 91.15\% \\ \hline JaSt & 93.00\% & 96.14\% & 92.03\% & 94.04\% \\ \hline ClamAV & 84.21\% & 95.01\% & 68.30\% & 79.47\% \\ \hline \hline \end{tabular}
\end{table} 
Table 9: The performance of the models on DB_DeOb. huge losses to Internet users, such as the stealing of credit card information. Therefore, any increase in accuracy will protect more Internet users from attackers. JaSt was proposed by [PERSON] et al. [13], which extracts features from the AST and uses the frequency analysis of specific patterns to detect malicious JavaScript. Since JsSt is open source, we use it to compare with our method. Table 9 shows the experimental results, the perform of the BLSTM neural network is better than JaSt with 4.71% higher accuracy, 5.88% higher recall rate, and 4.25% higher F1-score. This means that the program slices can extract more semantic information than simply traversing the AST, and the increase in semantic information improves the performance.