# Helping CNAs Generate CVSS Scores Faster and More Confidently Using XAI
## 7 Discussion
### SHAP Limitation for Text Datasets
\begin{table}
\begin{tabular}{c c c c} \hline
**Word** & **Occurrence** & **Word** & **Occurrence** \\ \hline vulner & 64,706 & savgsanit & 1 \\ \hline attack & 51,580 & textnot & 1 \\ \hline allow & 37,937 & htmlpurifi & 1 \\ \hline version & 36,899 & dataobject & 1 \\ \hline user & 29,445 & 210p23 & 1 \\ \hline \end{tabular}
\end{table} 
Table 13: Occurrence of the five most (left) and least (right) common words. To make the SHAP scores more meaningful, we propose a simple linear equation that normalizes the scores using global and in-class frequencies of the words. The equation, depicted below, involves multiplying the initial word score by the word frequency in the class divided by the total frequency of the word. This normalization process generates a simple normalized score that can be utilized for further analysis. We believe that this approach can be improved upon by the research community to enhance the interpretability of SHAP scores. \[normalized\_score=score\times\left(\frac{class\_frequency}{total\_frequency}\right) \tag{2}\]  
We incorporated the normalized score for each word in Table 14 under the \"Normalized Score\" column. With the introduction of the normalized score, we observe a shift in word importance. For instance, \"intel\" now emerges as the most important word for the \"local\" class, surpassing all other competitors by a factor of five. The inclusion of the normalized score enables us to identify outlier words that exhibit high scores but low frequencies, as well as words that possess both high scores and high frequencies. Additionally, the range between the minimum and maximum values of the scores generally decreased for all classes except for the \"local\" class. This normalization process aids in detecting words that have distinctive importance relative to the rest, regardless of their frequency or score. It provides a more balanced perspective on the significance of each word within its respective class. By incorporating the normalized score, we can effectively identify words that have an exceptional impact and stand out from the rest. This enables a more nuanced understanding of the importance of each word in the context of the CVSS prediction problem. While our proposed solution offers a temporary fix, we advocate for further research in the realm of text classification explanation methods that specifically address challenges inherent to text datasets. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Word** & **Class** & **Score** & **Frequency** & **Frequency in Class** & **Normalized Score** \\ \hline certificate & Local & 168.913 & 3660 & 31 & 1.43 \\ \hline intel & Local & 100.732 & 1684 & 836 & 50.01 \\ \hline response & Local & 100.283 & 2035 & 43 & 2.12 \\ \hline office & Local & 96.875 & 2471 & 162 & 6.35 \\ \hline segment & Adj. network & 303.192 & 293 & 94 & 97.27 \\ \hline fi & Adj. network & 295.831 & 660 & 86 & 38.55 \\ \hline lan & Adj. network & 279.997 & 406 & 59 & 40.69 \\ \hline recommended & Adj. network & 122.748 & 945 & 16 & 2.08 \\ \hline scripting & Network & 995.643 & 20,375 & 7184 & 351.05 \\ \hline csf & Network & 523.327 & 4436 & 2049 & 241.73 \\ \hline expression & Network & 106.504 & 790 & 423 & 57.03 \\ \hline smartphone & Network & 95.693 & 273 & 204 & 71.51 \\ \hline physically & Physical & 592.882 & 529 & 74 & 82.94 \\ \hline lock & Physical & 354.887 & 614 & 71 & 41.04 \\ \hline screen & Physical & 193.865 & 842 & 63 & 14.51 \\ \hline lockscreen & Physical & 176.406 & 55 & 24 & 76.98 \\ \hline \hline \end{tabular}
\end{table} 
Table 14: Word affinities, global word frequencies, in-class word frequencies, and normalized scores for the Attack Vector label’s four classes, ordered from highest to lowest affinity.