# Automated Software Vulnerability Detection Based on Hybrid Neural Network
### Experiments for Answering RQ3  
In order to answer RQ3, we compared our proposed method with five state-of-the-art vulnerability detection methods. \"Flawfinder\" [46] is a famous open-source vulnerability detector using static analysis. \"Checkmarx\" [3] is a commercial tool based on dependency analysis. \"Flawfinder\" and \"Checkmarx\" represent traditional vulnerability detection methods. Both of them rely on the vulnerable programming patterns defined by experts. \"SySeVr\" [15] is a source code-based detection method. Program slice is applied to generate candidates, and a Bi-GRU network is utilized to learn vulnerable programming patterns automatically. \"VulDeeLocator\" [41] is an automatic method that uses sliced LLVM IR as candidates and utilizes Bi-GRU and attention mechanisms to learn vulnerable programming patterns. Both \"SySeVr\" and \"VulDeeLocator\" are sequence-based methods. \"Devign\" [26] is a graph-based detection method. The samples were transformed into a graph representation, and then GGRN was applied to learn vulnerable programming patterns automatically. The results are listed in Table 5.  
As shown in Table 5, compared with vulnerable pattern learning methods, the traditional approaches \"Flawfinder\" and \"Checkmarx\" had low detection performance. Their poor performance was because traditional methods detect vulnerabilities based on expert-defined rules, which cannot fit all situations. \"SySeVr\" achieved a 37.5%, 45.4%, 40.3%, and 43.4% improvement over \"Checkmarx\" in accuracy, precision, recall, and F1-score, respectively. This improvement demonstrates the ability of the sequence-based machine learning method. \"VulDeeLocator\" achieved an 8.6%, 14.7%, 7.4%, and 11.1% improvement over \"SySeVr\" in accuracy, precise, recall, and F1-score, respectively. This improvement demonstrates the potential of LLVM IR for vulnerability detection. \"Devign\" achieved a slight improvement over \"SySeVr\", which was because the application of graph representation learning highlighted the structure information. However, \"Devign\" takes a statement  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Model** & **Acc (\%)** & **P (\%)** & **R (\%)** & **F1 (\%)** & **TTime (s)** & **DTime (s)** \\ \hline Text-CNN & 92.6 & 95.6 & 89.8 & 92.6 & 11.0 & 1.5 \\ Bi-GRU & 98.2 & 96.5 & 98.8 & 97.6 & 203.6 & 109.8 \\ GRU + Attention & 93.8 & 90.5 & 93.2 & 91.8 & 831.2 & 180.9 \\ Our model & 99.0 & 98.3 & 99.0 & 98.6 & 1086.7 & 122.1 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Results for RQ2. Comparison of 4 different neural networks.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Method** & **FPR (\%)** & **FNR (\%)** & **Acc (\%)** & **P (\%)** & **R (\%)** & **F1 (\%)** \\ \hline Flawfinder [46] & 12.9 & 80.3 & 62.1 & 47.3 & 19.7 & 27.8 \\ Checkmarx [3] & 46.5 & 51.8 & 51.6 & 37.7 & 48.2 & 42.3 \\ SySeVr [15] & 10.6 & 11.5 & 89.1 & 83.1 & 88.5 & 85.7 \\ VulDeeLocator [41] & 1.3 & 4.1 & 97.7 & 97.8 & 95.9 & 96.8 \\ Devign [26] & 8.3 & 13.3 & 89.8 & 86.0 & 86.7 & 86.4 \\ Our method & 1.0 & 1.0 & 99.0 & 98.3 & 99.0 & 98.6 \\ \hline \hline \end{tabular}
\end{table}
Table 5: Results for RQ3. Comparative analysis.  
as the base element of a graph. Each statement is obtained by calculating the average of all inside tokens. This operation will lose the local details, which is crucial for vulnerability detection. Our method achieved 99.0%, 98.3%, 99.0%, and 98.6% in accuracy, precision, recall, and F1-score, respectively, reaching a new state-of-the-art on the SARD dataset. This is attributed to the proposed intermediate representation and the hybrid neural network, which takes both long-term dependencies and local details.