# Helping CNAs Generate CVSS Scores Faster and More Confidently Using XAI
## 7 Discussion
### Exclusive Words Based on Statistical Presence  
Building on the insights from the previous subsection, we sought to explore the effectiveness of constructing an IF rule for exclusive words based on their actual presence in the text. Our rationale behind this exploration is rooted in the notion that if certain vulnerabilities consistently exhibit specific keywords, there may be value in simplifying the classification process for analysts. Moreover, adopting this approach could potentially reduce the complexity of the dataset for the ML model, enhancing its performance. Furthermore, an IF rule based on the statistical occurrence of a word within the dataset, associated with an exclusive class, offers a transparent, straightforward, and informative explanation for a subset of the data. This approach is easily updatable, is subject to scrutiny, and empowers analysts to directly apply their insights to future vulnerability analyses. 1. Extracting occurrence-based exclusive words: * Define a dictionary to store words and their associated classes. * Iterate over all the descriptions of the dataset. * For each description, iterate over all its words and append the class of its description to the dictionary. * Keep only the word keys that have a single class value, as these are our exclusive words. 2. Assigning Designated Classes: * Iterate over all the descriptions. * For each description, search for exclusive words using the previous dictionary. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Method** & **Accuracy** & **F1** \\ \hline Random pick & 0.372 & 0.321 \\ \hline Highest single & 0.435 & 0.389 \\ \hline Highest sum & **0.537** & **0.528** \\ \hline \hline \end{tabular}
\end{table} 
Table 15: Experimental results of generated rules. Highest results are highlighted in bold. When an exclusive word is found, note down its designated class. * If multiple exclusive words are identified, record the associated class only if it remains consistent for all words; otherwise, skip the description. It is important that since this method only extracts words that are exclusively seen with a single class each, therefore, classifying those descriptions using the designated class will always yield, by design, correct predictions. In this case, the metric we need to focus on is the data coverage (the portion of the dataset to which this rule applies). In the end, the data coverage for this approach is 46.67%, amounting to 38,217 vulnerabilities. This means that almost half the dataset can be classified using this approach. As this is an important number, this approach could function as an initial filter, with only vulnerabilities lacking exclusive words proceeding to the ML model. Additionally, since the method is straightforward, we can precisely identify which word(s) helped reach that decision, achieving our goal of both a clear explanation and high precision. CNA analysts may establish a streamlined process wherein each new vulnerability description undergoes a check for the presence of occurrence-based exclusive words. If such words are identified, the vulnerability is classified based on the predicted class. In instances where no exclusive words are found, the description is forwarded to an ML model, which generates a classification result along with a confidence score. The team can set a confidence score threshold, e.g., 90%, and if the model surpasses this threshold, the predicted class is automatically assigned. For cases where neither the IF rule nor the model can provide a classification, human analysts can intervene. The workflow of this pipeline is illustrated in Figure 17.  
To add more transparency, in the case that the IF rule does find exclusive words, we can highlight them in a plot to make it easier for analysts to identify them. To enhance the visual presentation, we can output the sentence with the exclusive words highlighted, as shown in Figure 18.  
Figure 17: CVE classification pipeline.