## 4 Experiments and Results
### Results  
Table 2 and Figure 8 show the results for the eight experimental patterns.  
RQ1 How Accurately Can a CVE-ID Be Traced to Its Associated CAPEC-ID following a Link between Cybersecurity Databases?  
Of the 61 CVE-IDs, only 4 were successfully traced. This low accuracy is due to the CVE-CWE link. First, some of the identified CVE-IDs are not linked to the CWE. Figure 9 shows the percentage of CVE-IDs that are linked to the CWE by year. Although 80% have been linked since 2008, 20% have not. This 20% cannot be traced to the CWE.  
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Recall@10** & **MRR** \\ \hline TF-IDF (document) & 0.787 & 0.591 \\ SBERT (section average) & 0.803 & 0.368 \\ TF-IDF (per section) & 0.770 & 0.596 \\ TF-IDF (section average) & 0.754 & 0.470 \\ USE (document) & 0.705 & 0.491 \\ USE (per section) & 0.672 & 0.420 \\ USE (section average) & 0.590 & 0.474 \\ SBERT (per section) & 0.557 & 0.337 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Results for the eight patterns using the proposed approach.  
Figure 8: Scatter plots of the results for the eight patterns using the proposed approach.  
Second, many CVE-IDs are linked to CWE-IDs with a high level of abstraction, such as CWE-20 and CWE-200. Figure 10 shows the frequency that highly abstract CWE-IDs are linked to each other. CWE-20 and CWE-200 are linked with high frequency. Highly abstract CWE-IDs generate two issues. First, the underlying weaknesses become invisible, and the chain and complex relationships are hidden. CWE-20 has the following description:  
Figure 10: Vulnerability distribution by CWE-ID.  
Figure 9: Changes in the percentage of CVE-ID linked to CWE-ID by year.  
The \"input validation\" term is extremely common, but it is used in many different ways. In some cases, its usage can obscure the real underlying weakness or otherwise hide chaining and composite relationships [42].  
Based on the above description, a highly abstract CWE-ID may not identify the root cause or trace the attack. Second, one CWE-ID is associated with many CAPEC-IDs. CWE-20 is linked to 51 CAPEC-IDs and CWE-200 is linked to 58 CAPEC-IDs. Hence, it is difficult to identify the correct CAPEC-ID. The relationship between CVE-ID and CWE-ID is not always considered as a characteristic of vulnerability usage by attackers.  
**RQ1. Answer**  
Only 4 out of the 61 CVE-IDs were traced to the associated CAPEC-ID. CAPEC was not taken into account in the link between CVE and CWE. Therefore, following CVE to the CAPEC attack information may not provide useful information.  
RQ2 How Accurate Is the Tracing of CVE-IDs to the Associated CAPEC-IDs When Using the Proposed Approach?  
Table 2 shows that the \"document\" pattern is better suited for TF-IDF and USE. The \"section average\" pattern is most suitable for SBERT. Figures 11-13 show the experimental results for these three patterns. Figure 14 shows a box-and-whisker diagram of average precision. All algorithms traced more than 70% of the CVE-IDs. The experiment involved only one ground truth because the link between CVE and CAPEC is a many-to-one link. This is the reason for the low precision. Changing the criteria of the ground truth impacts the value of precision. The number of words in the CVE description and CAPEC document are not correlated with the success rate of the proposed approach. A characteristic of CVE-IDs that failed was an insufficient CVE description. This is due to missing cybersecurity words important to trace or hidden fundamental weaknesses. Figure 15 shows the results of a Kruskal-Wallis test. The \(p\)-value less than 0.05 indicates a significant difference.  
Figure 11: Recall for each approach.  
Figure 12: Precision for each approach.  
Figure 13: F-measure for each approach.  
**RQ 2. Answer**  
All three algorithms were successful in tracing. All algorithms were 70% successful. In order of success rate, SBERT successfully traced 49 CVE-IDs, TF-IDF traced 48 CVE-IDs, and USE traced 43 CVE-IDs.