# Helping CNAs Generate CVSS Scores Faster and More Confidently Using XAI
## 4 Methodology
### Dataset  
To obtain up-to-date data, we utilize the official API provided by the US National Vulnerability Database (NVD). This API allows us to retrieve vulnerabilities from the time of creation of the database up until the current date. The results are obtained in JSON format, but the structure has undergone several changes over the years. To handle this, we developed a data transformation pipeline that flattens the nested dictionaries and retains the most relevant information required for our analysis. While we are only interested in the vulnerability descriptions and the target metric values, we kept and transformed the whole dataset to benefit more researchers upon public release. Currently, only [PERSON] et al. (2020) have made their dataset publicly available, which includes 152,000 vulnerabilities up to April 2021, comprising only the description and target metric values.  
Our data scraping was conducted on 20 March 2023, when the NVD had a total of 210,101 CVEs. We scraped all available data excluding vulnerabilities with missing or empty information. The final dataset encompasses 197,755 CVEs published between October 1988 and March 2023, including all top-level columns and several second- and third-level columns. The distribution of the CVEs based on their status is summarized in Table 5. For our analysis, we utilized the \"Analyzed\" and \"Modified\" CVEs, as they contain all the necessary information we required, so 93.8% of the data.  
Additionally, as the goal of this paper is to develop a tool for CNA analysts to enhance their efficiency in analyzing CVE vulnerabilities, we made a deliberate decision not to include any external data to maintain the data distribution focused only on real CVE data and avoid introducing bias, ensuring that it is trained on the same type of examples analysts will encounter in the future.  
Table 6 presents the distribution of CVEs based on their version. It is worth noting that the total count may exceed the number of CVEs due to several vulnerabilities having multiple version scores. While it is technically possible to convert CVEs from version 3.0 to 3.1, we encountered numerous inconsistencies in the sources we reviewed. As a result, we decided to focus exclusively on version 3.1 for our analysis.  
Figure 8 provides an overview of the distribution of severities in the dataset. It is worth noting that the low-severity class appears to be under-represented compared to the other severity levels. Since these vulnerabilities do not pose as much of a problem as the rest, there is no need to deal with the data imbalance in this context.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **V2** & **V3.0** & **V3.1** & **V3.X** \\ \hline Count & 179,758 & 52,605 & 81,889 & 124,322 \\ \hline \hline \end{tabular}
\end{table}
Table 6: CVEs distribution based on version.  
\begin{table}
\begin{tabular}{l c c} \hline \hline
**Status** & **Count** & **Percentage** \\ \hline
**Total** & 210,101 & 100\% \\ \hline Analyzed & 126,055 & 60\% \\ \hline Modified & 71,100 & 33.8\% \\ \hline Rejected/deferred & 12,346 & 5.9\% \\ \hline Undergoing analysis & 408 & 0.2\% \\ \hline Awaiting analysis & 165 & 0.08\% \\ \hline \hline \end{tabular}
\end{table}
Table 5: CVEs distribution based on status.  
Figure 9 illustrates the distribution of vulnerability descriptions in the dataset. Most descriptions fall under the 500-word mark, with a normal distribution centered around 250 words and some outlier descriptions that are over 1000 words long. We considered this when choosing a preprocessing technique and ML model.  
Finally, Figure 10 indicates that the label distribution for each metric is very different, with some metrics suffering from severe class imbalance. We also consider this in the following sections.  
Figure 8: The severity distribution of the CVE data.  
Figure 9: Length distribution of vulnerability descriptions.