# A Context-Aware Neural Embedding for Function-Level Vulnerability Detection
## 3 Experiments and Evaluation
### Result Analysis
#### 3.4.4 Detection of Context-Related Vulnerabilities  
Table 5 presents the results of the number and the proportion of CWE-119 vulnerabilities accounted for all the found vulnerabilities in top-\(k\) on projects: FFmpeg, LibTIFF, and OpenSSL. As Table 5 shows, there are 47 CWE-119 vulnerabilities in the test set of FFmpeg, accounting for approximately 33% of the total vulnerabilities in the test set. When returning 10 of the most probable vulnerable functions, there are 9 actually vulnerable ones, and 4 of them are CWE-119 vulnerabilities, which account for 44%. When returning 20 of the most probable vulnerable functions, there are 18 actually vulnerable functions, and 10 of them are CWE-119 vulnerabilities, accounting for 56% of the total vulnerable functions. Similarly, when returning 30, 40, and 50 functions, the proportions of CWE-119 vulnerabilities are 50%, 48%, and 44%, respectively, which all exceed 33%. This indicates that our proposed framework is more likely to detect CWE-119 vulnerabilities that are context related.  
For projects LibTIFF and OpenSSL, similar results are observed. As Table 5 shows, in most of the cases, our framework can detect more CWE-119 vulnerabilities, except for the case on LibTIFF where 20 functions are returned and 15 are actually vulnerable, among which 10 vulnerable functions belong to the CWE-119 category, accounting for 67% of the total. For other cases, our framework can detect more CWE-119 vulnerabilities than the other types, which is indicated by the proportions of the CWE-119 in the found vulnerabilities being higher than the proportion of the CWE-119 in all vulnerabilities in the test set.  
\begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Software** **Project**} & \multirow{2}{*}{**Detection**} & \multicolumn{6}{c}{**\# of Vulnerable Functions**} & \multicolumn{2}{c}{**Found in Top-\(k\) (Top-\(k\) Precision)**} \\ \cline{3-10}  & & **Top 10** & **Top 20** & **Top 30** & **Top 40** & **Top 50** & **Top 100** & **Top 150** & **Top 200** \\ \hline \multirow{10}{*}{FFmpeg} & Our framework & \multirow{2}{*}{9 (90\%)} & Our framework & \multirow{2}{*}{18 (90\%)} & Our framework & \multirow{2}{*}{24 (80\%)} & Our framework & \multirow{2}{*}{31 (78\%)} & Our framework & \multirow{2}{*}{61 (61\%)} & Our framework & \multirow{2}{*}{65 (43\%)} & Our framework & \multirow{2}{*}{72 (36\%)} \\  & + RF & & & & & & & & & \\ \cline{1-1} \cline{5-10}  & Our framework & \multirow{2}{*}{9 (90\%)} & Our framework & \multirow{2}{*}{16 (80\%)} & Our framework & \multirow{2}{*}{19 (63\%)} & Our framework & \multirow{2}{*}{19 (63\%)} & Our framework & \multirow{2}{*}{19 (58\%)} & Our framework & \multirow{2}{*}{19 (58\%)} & Our framework & \multirow{2}{*}{19 (58\%)} & Our framework & \multirow{2}{*}{19 (58\%)} \\  & + SVM & & & & & & & & & \\ \cline{1-1} \cline{5-10}  & Cross-VD [14] & 7 (70\%) & 9 (45\%) & 14 (47\%) & 17 (43\%) & 18 (36\%) & 30 (30\%) & 39 (26\%) & 46 (23\%) \\ \cline{1-1} \cline{5-10}  & Multi-VD [27] & 9 (90\%) & 14 (70\%) & 20 (67\%) & 24 (60\%) & 28 (56\%) & 47 (47\%) & 60 (40\%) & 70 (35\%) \\ \cline{1-1} \cline{5-10}  & DeepBalance[38] & 9 (90\%) & 15 (75\%) & 20 (67\%) & 26 (65\%) & 33 (66\%) & 61 (61\%) & 63 (42\%) & 71 (36\%) \\ \cline{1-1} \cline{5-10}  & Flawfinder & \multicolumn{6}{c}{Found 44 vulnerable functions, but not in the Top 200 list} \\ \hline \multirow{10}{*}{LibTIFF} & Our framework & \multirow{2}{*}{7 (70\%)} & Our framework & \multirow{2}{*}{15 (75\%)} & Our framework & \multirow{2}{*}{22 (73\%)} & Our framework & \multirow{2}{*}{25 (63\%)} & Our framework & \multirow{2}{*}{41 (41\%)} & Our framework & \multirow{2}{*}{50 (33\%)} & Our framework & \multirow{2}{*}{19 (58\%)} & Our framework