# Automated Software Vulnerability Detection Based on Hybrid Neural Network
### Experiments for Answering RQ2  
Experiments were performed to compare our method with three embedding methods to verify our method's embedding effect. Specifically, \"vocabulary\" represents the local embedding method, which utilizes the vocabulary model directly to encode the inputs. A 4-layer fully connected neural network was applied to generate the detection results. The last three methods are distributed embedding methods. They use our hybrid neural network as a representation learning model. \"Random\" denotes the embeddings of inputs randomly initialized. \"Static\" denotes the embeddings of inputs generated by word2 vec in the pre-training stage. Furthermore, the embedding matrices remain static in the training stage. \"None-static\" denotes the embeddings of inputs generated by word2 vec in the pre-training stage. The embedding matrices were modified in the training stage. The results are shown in Table 3.  
As shown in Table 3, the distributed embedding method had a significant improvement in vulnerability detection effectiveness compared with the local embedding method. Using the same neural network, \"random\" achieved the best F1-score, recall, accuracy, and false negative rate. \"None-static\" achieved the best performance on false positive rate and precision. In general, \"random\" achieved the best embedding effect on the SARD dataset.  
To evaluate our proposed hybrid neural network's ability, experiments were performed to compare our method with three neural networks, which have been proven effective in processing sequence data. All experiments were conducted on the same dataset processed by our method discussed in Section 3.3. Neural networks for comparison were Text-CNN, Bi-GRU, and GRU + Attention. They all use the \"random\" embedding method. The results are shown in Table 4. Six indicators were utilized to measure the effectiveness of a network. Specifically, \"Acc\", \"P\", \"R\", and \"F1\" are described in Section 4.2. \"TTime\" denotes the training time that neural networks cost in an epoch. \"DTime\" denotes the  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Method** & **FPR (\%)** & **FNR (\%)** & **Acc (\%)** & **P (\%)** & **R (\%)** & **F1 (\%)** \\ \hline Vocabulary & 12.9 & 80.3 & 84.1 & 79.7 & 73.8 & 76.6 \\ Random & 1.0 & 1.0 & 99.0 & 98.3 & 99.0 & 98.6 \\ Static & 1.4 & 3.6 & 87.8 & 97.6 & 96.4 & 97.0 \\ None-static & 0.9 & 3.0 & 98.3 & 98.4 & 97.0 & 97.7 \\ \hline \hline \end{tabular}
\end{table}
Table 3: Comparison of 4 different embedding approaches.  
detecting time of the trained model on 9858 samples. Target samples and training samples are different vulnerabilities and are both from the SARD dataset.  
As shown in Table 4, \"Text-CNN\" had a minimum training time of 11.0 s/epoch and a detecting time of 1.5 s. However, it had the lowest accuracy and recall. \"Bi-GRU\" made a significant improvement compared with \"Text-CNN\". \"GRU + Attention\" had the worst performance on precise and F1-score. Our proposed hybrid neural network achieved the best results on accuracy, precision, recall, and F1-score, which were 99.0%, 98.3%, 99.0%, and 98.6% respectively. Although our model took 1086.7 s/epoch to train the network, the improvement was clear. Vulnerability detection aims to discover potential vulnerabilities as much as possible. Therefore, the training time for our method is acceptable.