# Automated Software Vulnerability Detection Based on Hybrid Neural Network
### Experiments for Answering RQ1  
In order to evaluate the effect of our proposed intermediate representation, experiments were conducted on five datasets. All five datasets were generated from the same dataset. Specifically, \"raw data\" denotes the raw dataset described in Section 4.3, which consists of programs with vulnerability and corresponding patches in their bodies. Moreover, a sample could include multiple source code files. \"LLVM IR\" denotes the intermediate codes that were transformed from \"raw data\". \"Source code based\" consists of program slices in the form of source code. It was transformed from \"raw data\" using our slice criterion-finding method. \"VulDeeLocator\" is a dataset that was generated by VulDeeLocator [42], which is also an LLVM IR and program slicing based method. Table 2 summarizes these five datasets from the number of samples, the average length of a sample, the vocabulary size, and the F1-score. The indicator F1-score obtained by applying the same neural network on these datasets represents the ability of intermediate representation to promote vulnerability detection. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & \multicolumn{2}{c}{**Description**} \\ \hline Embedding layer & The size of vocabulary and token vector are 538 and 100, respectively. The embedding matrix is randomly initialized. The max sequence length is 1000. \\ \hline Convolutional layer & The number of convolutional filters is 256 with size 3. The window of the max-pooling filter is 4. \\ \hline Recurrent layer & The units of both forward and backward GRU net are 256. Dropout and recurrent_dropout are set to 0.2 and 0.1, respectively. \\ \hline Dense layer & The first two dense layers with 256 units are connected to the convolutional layer and the recurrent layer. The last dense layer generates prediction with two units. \\ \hline Batch_size & The number of samples that are propagated through the network is 128. \\ \hline Loss function & The function to evaluate the difference between the trained model and datasets is binary_crossentropy. \\ \hline Optimizer & The algorithm to optimize the neural network is Adam. \\ \hline Monitor & The matric to be monitored for early stop is F1-score and patience is 10. \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Tuned parameters of hybrid neural network. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Data** & **Sample Number** & **Average Length** & **Vocabulary Size** & **F1-Score** \\ \hline Raw data & 12,301 & 185 & 60,918 & - \\ LLVM IR & 12,301 & 513 & 43,179 & - \\ Source code based & 32,860 & 144 & 691 & 88.9 \\ VulDeeLocator [42] & 140,631 & 34 & 2665 & 97.4 \\ Our method & 32,860 & 51 & 537 & 98.6 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Results for RQ1. Evaluating the performance of different intermediate representations. As shown in Table 2, \"raw data\" had 12,301 samples in total. The average length of a sample was 185, and the vocabulary size was 60,918. After compiling, the average length increased to 513 in \"LLVM IR\", because LLVM IR was in the form of an SSA, and non-representative codes were introduced. Both \"raw data\" and \"LLVM IR\" had no F1-score because vulnerability and patch were mixed in some samples. As for \"source code based\", 32,860 samples were extracted from \"raw data\" using our proposed slice criterion finding method. The average length of a sample and the vocabulary size were reduced to 144 and 691, respectively, after normalization. \"VulDeeLocator\" had 140,631 samples, which is approximately four times that of \"our method\", because \"VulDeeLocator\" travels the entire program to find all security-sensitive points whose bidirectional program slice includes a vulnerable line given in the vulnerability description file. This method will incur a situation in which multiple samples may belong to one vulnerability. The generated slices are inaccurate and fragmented. Moreover, \"VulDeeLocator\" normalizes custom-defined function names, reducing the vocabulary size from 43,197 to 2665.